(07:59:11) peter_grehan: finally logged on...
(07:59:36) brycedenney: hey
(07:59:41) brycedenney: how's it going?
(08:00:28) brycedenney: if lag is too bad, we can try irc or AOL IM
(08:01:40) peter_grehan: sorry
(08:01:46) peter_grehan: fire away with the net questions.
(08:02:07) brycedenney: did you see Dave's post this morning?
(08:02:25) peter_grehan: no
(08:02:27) brycedenney: (EKS) he and I got to talking about kernel drivers
(08:03:05) brycedenney: you'll get it soon, I guess we should start with the big picture
(08:03:30) brycedenney: we need bochs to be able to talk to the host machine and other machines in the world.
(08:03:43) peter_grehan: just reading the email now...
(08:03:46) brycedenney: ok
(08:04:14) brycedenney: we have user-code solutions for a few OSes that talk to the world but not the host machine
(08:04:57) peter_grehan: ok
(08:05:32) brycedenney: so we need to dream up a plan to talk to both host and world, eventually supporting all systems
(08:05:36) brycedenney: :)
(08:06:36) peter_grehan: this involves writing a virtual ethernet driver
(08:06:56) brycedenney: isn't that what EKS describes?
(08:07:09) peter_grehan: engineer the interface from the source they provide. However
(08:07:24) peter_grehan: yep
(08:08:31) peter_grehan: the host o/s driver *can't* be generic
(08:08:34) peter_grehan: ps-specific code
(08:08:40) brycedenney: os
(08:08:43) brycedenney: right
(08:09:38) brycedenney: ok so one plan is to write a virtual ethernet driver for each supported OS
(08:10:13) peter_grehan: that's the only real way to do this.
(08:10:15) brycedenney: I need to test this: can you get multiline sends (this is 3 lines long)
line 2
line 3
(08:10:51) peter_grehan: yep. 
(08:10:53) brycedenney: good
(08:11:02) brycedenney: one plan:
(08:11:06) brycedenney: A virtual ethernet driver allows you to talk to the host and the world.
Pros: talk to both host and world
Cons: Must have root access.

(08:11:23) brycedenney: does that seem correct to you?
(08:11:41) brycedenney: cons: must be written separately for each OS
(08:12:32) peter_grehan: not necessarily - it really only allows you to talk to the host
(08:12:42) brycedenney: but then IP routing lets you talk to the rest
(08:12:49) brycedenney: without any wierd hacks
(08:12:55) brycedenney: ?
(08:13:20) peter_grehan: only if forwarding is enabled
(08:13:38) brycedenney: with root access, you can make it work
(08:13:45) peter_grehan: yep.
(08:13:48) brycedenney: you might need to redo your kernel or something
(08:14:11) peter_grehan: you need to load a module
(08:15:08) brycedenney: how about this then:
(08:15:11) brycedenney: 1. A virtual ethernet driver
Pros: you can talk to both host and world
Cons: Must have root access to load a kernel module,
must be written separately for each host OS,
must set up IP forwarding to talk to the world.

(08:15:56) peter_grehan: that's correct.
(08:16:05) brycedenney: ok here's another plan
(08:16:21) brycedenney: 2. packet filters (BPF, LSF)
Pros: infrastructure is already there in iodev/eth_*.cc
each packet mover can support several OSes if
they have the right library/kernel feature.
user can run it,
Cons: can't talk to host

(08:17:44) peter_grehan: that's right. there are still some issues with this - at one stage
(08:18:33) peter_grehan: I just remembered: there is a port of BPF to windows which I've used. It's quite good
(08:18:46) brycedenney: cool!
(08:19:39) peter_grehan: one other option is a kernel module that hooks into bridging code. this is what VmWare uses - it makes the guest o/s look like it's on the same LAN as the host o/s. Very similar to the packet filter option.
(08:20:35) brycedenney: does bridging code make other machines on the net see a different MAC address from the host?
(08:20:50) brycedenney: or is it IP based?
(08:21:08) peter_grehan: yep
(08:21:12) brycedenney: ip
(08:21:15) brycedenney: ?
(08:22:16) peter_grehan: I should have mentioned: other hosts would see a different MAC addr
(08:22:24) brycedenney: cool!
(08:22:49) brycedenney: any hope of doing that in windows? does it have bridging code?
(08:23:31) peter_grehan: no
(08:23:49) brycedenney: but it's a good solution for systems that support it
(08:24:28) peter_grehan: it's the best in that it requires no extra config on the host o/s
(08:25:32) brycedenney: 5. Kernel module using bridging code
Pros: bochs appears to be a different MAC than the host,
requires no extra config on the host o/s
Cons: Must have root access to load a kernel module,
must be written separately for each host OS,
bridging code not present in all kernels,

(08:26:47) peter_grehan: yep
(08:27:01) brycedenney: you may wonder where are 3 & 4 :)
(08:27:08) brycedenney: 3. TCP only networking
Pros: can be portable socket code,
can run as a normal user
Cons: each protocol must be handled separately
hard to extract TCP information from ethernet frames (maybe
we could borrow linux or other networking code)

(08:27:58) peter_grehan: that one ain't gonna work. there are whole companies built on the basis of less code than what that will take !
(08:28:10) brycedenney: :)
(08:28:47) brycedenney: I don't want to spent too much time on it
(08:29:17) brycedenney: but why couldn't you rip off code from an open source kernel
(08:29:30) brycedenney: that translates ethernet frames into a usable form
(08:30:11) peter_grehan: it's not just translating - you have to implement a complete stack
(08:30:21) peter_grehan: possible work for outgoing TCP connections
(08:30:34) peter_grehan: I should say
(08:32:02) brycedenney: You mentioned in an email that you have to get the kernel code running as a user, and FYI ther's a whole project that has already done that (http://user-mode-linux.sourceforge.net/)
(08:33:24) brycedenney: for incoming TCP connections you would need some tricks. What if you tell bochs "Map local TCP port 3023 to bochs port 23"
(08:34:02) brycedenney: then bochs would have to bind to TCP port 3023, and when it gets connections it would build up the appropriate ethernet frame and put it into bochs's NE2K packet mover.
(08:35:01) peter_grehan: yep
(08:35:01) peter_grehan: psuedo driver !
(08:35:01) peter_grehan: plus
(08:35:30) brycedenney: The only reason I'm so interested in this plan is the pros are very important: portable code, running as a user
(08:35:44) peter_grehan: that's a lot of configuration. you may not know in advance all the ports you would need.
(08:35:51) brycedenney: almost all the other plans, a student with a user account could never even try it.
(08:36:15) brycedenney: bochs configuration at runtime we can do, we're planning a control panel of sorts
(08:37:03) brycedenney: we linux users have to remember, not everybody is root! :)
(08:37:58) peter_grehan: the student account thing isn't quite true
(08:38:13) brycedenney: what plan would allow non-root networking?
(08:39:41) peter_grehan: I can't think of one that is going to cover all cases. For windows
(08:41:58) peter_grehan: I'll try and draw some pictures and send them to you
(08:42:01) brycedenney: ok
(08:42:26) brycedenney: so under cons I will add "Peter thinks it's not worth the trouble" :)
(08:42:44) brycedenney: BTW, how much time do you have left?
(08:42:55) peter_grehan: yeh
(08:43:22) peter_grehan: maybe another 1/2 hour or so... did you want to go over any other net stuff ???
(08:43:34) brycedenney: I have one more plan.....
(08:43:37) brycedenney: :_
(08:43:39) brycedenney: :)
(08:43:52) peter_grehan: have at it..
(08:44:00) brycedenney: 4. IP only networking through hack of PPP on host. Bochs guest OS runs an
NE2K, bochs emulator includes some parts of user-level PPP code and moves
packets to/from the PPP kernel module.

Pros: PPP kernel code already written for many platforms
PPP user code already written
maybe can be run as a normal user
all IP protocols supported
Cons: maybe hard to extract IP information from ethernet frames
(is it really that hard?)
have to emulate some ethernet functions

(08:45:22) brycedenney: It's a bit crazy, but motivated by the fact that tons of people get all their network access through PPP, so it's not that much of a limitation for Bochs to get its network access through PPP.
(08:45:32) peter_grehan: I thought that would be better off done through the serial driver itself
(08:45:41) peter_grehan: doing a type of eth-ppp bridge.
(08:45:59) brycedenney: it seems like better performance to get many bytes at a time in the ne2k than sending bits at a time via serial
(08:46:11) peter_grehan: The serial driver is currently optimised for interactive input
(08:47:20) brycedenney: Do you have a concept of how many emulated instructions are required to build an ethernet frame VS sending the equivalent amount of data through serial driver?
(08:47:42) peter_grehan: Maybe I'm being a bit too harsh. Many early PPP implementations were made to look like ethernet drivers to the host. I just noticed today that the standard 'pppd' has a config option to direct output to a pseudo-terminal
(08:47:44) brycedenney: it's the number of emulated instructions that's the limiting factor.
(08:48:51) peter_grehan: The ne2k has to manually write every byte out in any case
(08:49:08) brycedenney: no DMA? :)
(08:49:27) peter_grehan: nope
(08:49:41) brycedenney: we could emulate a DMA capable card one day, but whatever.
(08:50:05) brycedenney: do you think serial and ne2k methods would take approximately the same # of bochs instructions to send out?
(08:51:00) peter_grehan: The serial is going to take more
(08:51:12) brycedenney: I expected it would, but why
(08:51:29) brycedenney: (I haven't programmed either one :) )
(08:51:58) peter_grehan: I only wanted to get a FreeBSD serial console working
(08:53:12) brycedenney: any reason why the current bochs serial emulation couldn't handle a PPP connection?
(08:53:54) peter_grehan: some work would be needed to abstract out the sending/receiving of bytes
(08:54:07) brycedenney: ?
(08:55:33) peter_grehan: the serial code currently writes bytes to file descriptor 0
(08:55:50) brycedenney: right, you'd want to open a TTY or something instead
(08:56:03) peter_grehan: yep.
(08:56:26) brycedenney: other than that, anything not supported that PPP might need?
(08:57:10) peter_grehan: You'd have to set up the TTY for raw mode
(08:57:45) brycedenney: ok, but as far as the serial port emulation in bochs, it has all the registers and bits
(08:58:31) brycedenney: Did you say why serial would take more instructions? I'm interested in some guess like 50% more, 100% more, since I don't know what the constraints are.
(08:58:34) peter_grehan: yep
(08:58:47) brycedenney: great! I'll play with serial a bit.
(08:59:11) brycedenney: (bad pun was not intended :) )
(09:00:20) peter_grehan: async PPP has overhead
(09:01:27) brycedenney: ok
(09:02:27) brycedenney: we didn't actually talk about
4. IP only networking through hack of PPP on host. Bochs guest OS runs an
NE2K, bochs emulator includes some parts of user-level PPP code and moves
packets to/from the PPP kernel module.
(09:02:52) brycedenney: except to say that it might be done with PPP on the guest os
(09:02:58) brycedenney: does it have any merits at all?
(09:03:19) brycedenney: is my description clear enough to know what I'm talking about?
(09:03:58) peter_grehan: Yeh
(09:04:01) peter_grehan: it could be made to work.
(09:04:23) brycedenney: it's only useful if the overhead of running PPP on the guest rather than ethernet driver is large enough
(09:05:06) brycedenney: if overhead is similar, we should just use serial since it doesn't require any PPP hacking.
(09:05:12) peter_grehan: it would also allow non-root networking
(09:05:55) brycedenney: same as running PPP direct, you may need root to set you up as a legal PPP user
(09:05:55) peter_grehan: it doesn't need hacking
(09:06:15) brycedenney: you mean plan 4 doesn't need hacking?
(09:06:57) peter_grehan: no kernel hacking
(09:07:35) brycedenney: right, but some changes to PPPD I expect
(09:08:34) peter_grehan: I don't think so - from what I saw
(09:09:00) brycedenney: are you saying bochs could read and write to a PPPD-controlled terminal
(09:09:27) brycedenney: and just translate ethernet<->PPP when bochs or the ppp had a packet?
(09:10:08) peter_grehan: That's right
(09:10:17) brycedenney: hey
(09:10:31) brycedenney: I didn't think of it that way, but it's good
(09:10:35) peter_grehan: Bochs serial driver - when a character is sent out
(09:10:57) peter_grehan: The translation from ethernet-ppp
(09:11:58) brycedenney: I missed something
(09:12:20) peter_grehan: about eth-ppp ?
(09:12:22) brycedenney: are you doing multiline sends ?
(09:12:33) peter_grehan: I think I hit return accidentally there :-(
(09:12:46) brycedenney: I only got: "Bochs serial driver - when a character is sent out" and then "The translation from ethernet-ppp" that's all
(09:13:13) peter_grehan: I'll type that bit again:
(09:13:21) brycedenney: sorry
(09:13:47) peter_grehan: when a character is sent out
(09:13:58) peter_grehan: did that come thru OK ?
(09:14:11) brycedenney: one line only "when a character is sent out"
(09:14:58) peter_grehan: next line: when a char arrives on the pseudo-term
(09:15:57) brycedenney: ok
(09:16:56) brycedenney: it seems that yahoo IM is failing here :(
(09:17:17) peter_grehan: the east coast is coming on-line :-)
(09:18:31) brycedenney: ping1
(09:18:57) peter_grehan: still here
(09:19:08) brycedenney: good
(09:19:16) brycedenney: I can't tell if I'm losing messages or just dense.
(09:19:38) peter_grehan: maybe msgs are being lost...
(09:20:04) brycedenney: we're close to being out of time anyway
(09:20:18) brycedenney: we have 6 plans now!!
(09:20:23) brycedenney: 1. A virtual ethernet driver

(09:20:27) brycedenney: 2. packet filters (BPF, LSF)
(09:20:31) brycedenney: 3. TCP only networking
(09:20:36) brycedenney: 4. IP only networking through hack of PPP on host.
(09:20:41) brycedenney: 5. Kernel module using bridging code
(09:20:45) brycedenney: 6. PPP link using bochs serial code
(09:21:00) brycedenney: all have pros and cons of course
(09:21:26) brycedenney: Is there any of them that you would like to work on, time permitting?
(09:22:12) brycedenney: or do you want to advise others who do have the time and seem to know what they're talking about?
(09:22:12) peter_grehan: I can help out
(09:22:19) peter_grehan: sure
(09:22:35) brycedenney: which seems most promising?
(09:22:42) brycedenney: what OS do you use anyway? freebsd?
(09:23:11) peter_grehan: I run Linux on my desktop
(09:23:18) brycedenney: cool, me too
(09:23:23) peter_grehan: I used a heavily modified FreeBSD when I was at Nokia :-)
(09:23:59) brycedenney: which plan interests you the most?
(09:24:37) peter_grehan: probably 1
(09:24:48) peter_grehan: 6 would be the easiest to get up and running.
(09:25:26) brycedenney: maybe it's worth making 6 work first since it seems pretty easy
(09:25:35) peter_grehan: sure
(09:25:53) brycedenney: and then let's start talking about #1 on bochs-developers and plex86 since they will both be interested
(09:25:58) brycedenney: esp on linux
(09:26:30) peter_grehan: OK. Is there much io-dev work coming out of plex86 ?
(09:26:47) brycedenney: we can probably get some volunteers to help
(09:27:17) brycedenney: iodev on plex86: not yet, but I think a lot of people are not sure how to help until Kevin's done more of the core stuff
(09:27:26) brycedenney: I think there will be interest.
(09:27:49) peter_grehan: they have some catching up to do
(09:28:08) brycedenney: plex86 doesn't consider kernel modules a Con :)
(09:29:04) peter_grehan: OK
(09:29:49) brycedenney: would you like to describe #1 in more detail and send it to plex86 and bochs-dev list? I'll finish up my summary of the various plans and send it to bochs-dev at least.
(09:30:10) peter_grehan: OK. See you later.
(09:30:12) brycedenney: and if you don't mind, I would like to post a log of this conversation since I think it will give some important details.
(09:30:22) peter_grehan: Sure.
(09:30:22) brycedenney: or put it on the web or something
(09:30:33) brycedenney: Thanks Peter! This was very helpful.
(09:30:43) peter_grehan: Bye.
(09:30:54) brycedenney: bye