Test Client?

[jordan]

Greetings Everyone...again. lol

With almost 3000 members, the amount of support we are having to provide to solve everyone's various networking issues is starting to put a strain on our time resources.  We understand that these problems usually exist within the user's domain due to router/firewall issues and a complete lack of understanding as to how this hardware works, or even what P2P gaming involves.

Russell mentioned in the faq/help about using the dxdiag to create a test session to the FSHost.

Would it be possible for someone who knows FSHost and directx to create a small executable test/diagnosis client?  Something that communicate with the server and tries to provide some feedback as to what ports are/aren't communicating correctly?  Also display the version of directx on the user's machine?

I can't emphasize enough, how helpful this could be.

If MSFS/FSHost users could run a stand-alone test client before starting any other troubleshooting steps..it could save everyone tons of time and frustration.  The FS just doesn't give enough feedback to the user.  It would also let a user know (on their own) that they have problems for sure, instead of all of the emails, posts, and private messages they usually send just to find out that their hardware is not configured properly for MSFS/FSHost p2p game play.

I'm crossing my fingers and toes that someone will agree that this is a good and time-saving idea.  Even though I am a programmer, I'm afraid I lack sufficient experience in this area to develop/implement a test client myself.

[Russell Gilbert]

This is a really interesting idea, and one that I'm not exactly sure how to do.  I already have a couple of test client programs I use for various things while working on FSHost, so I have a basis already.  But I think we need to work out exactly what this program would do.

I'm trying to think of all the cases I've run across with networking problems, and figure out how I'd solve them with this test program.  So help me out here...

First, we have to decide if we're going to support 2002 also, or just 2004.  A quick glance at FSHostSpy shows 13 of 43 players using 2002 (30%).  But I guess the real question is how often we have to deal with support issues related to 2002.  I should also say that 2004 is a lot easier to deal with (programmatically, anyway), since it uses fewer ports, and the ports are more predictable.

Second, we have to figure out which troubleshooting scenarios we'll support.  Here are a couple I can think of for 2004 (I'm sure there are others as well)...

1. FS sends a packet from its own port 23456 to FSHost's port 23456, but the response doesn't make it back to FS.  Can happen if a router on the FS machine is forwarding the port to a different machine on the local LAN.

How to detect it:  send a packet from own 23456 to FSHost's 23456.  Find out somehow that FSHost received it and the reply didn't make it back to us.  Suggest that the user check that 23456 is not already forwarded to another machine on the local LAN.  Or...  Find out somehow that FSHost never received the request.  Suggest that the user check to see if something (?) is blocking outgoing port 23456.

2. FS sends a packet to FSHost's 23456, but FSHost is hosting on 23455, so FSHost doesn't receive it, and doesn't reply.  In this case, FS also sends a packet to FSHost's 6073 (this is the way DirectPlay works in FS2004 -- 6073 is the DirectPlay default port, so it's always used in addition to whatever the FS user specifies).  FSHost replies to this packet, but it sends it back to FS's port somewhere in the 2300-2400 range.  (This is also because of the way DirectPlay works)  If the FS user is in a NAT situation, the incoming packet on port 23xx is blocked by the NAT server.  If the NAT is a router, the port range can be forwarded to fix the problem.  If the NAT is an NT/2003 server, it can't (not easily anyway), and the solution is to change FS to use the same port as FSHost (23455), so that when the reply is sent back, it'll be sent back to the same port FS originally sent the request from (23455) and the NAT will automatically forward it.  (Note that this issue of the reply being sent back on a different port in the 2300-2400 range only happens when 6073 is involved -- if FS sends the request on the same port that FSHost is using, the reply comes back on the same port)

How to detect it:  send a packet from own 23456 to FSHost's 23456.  Find out somehow that FSHost never received it, and that it's hosting on 23455.  Then send a packet from own 23456 to FSHost's 6073.  Find out somehow that FSHost replied to it on FS's port 2350, but that the test program never received the reply.  Suggest that the user change FS to use 23455, or to forward 2300-2400.  Or...  find out somehow that FSHost never received the request.  Suggest that the user check to see if something (?) is blocking outgoing ports 23456 and 6073.

So the key problem here is how to know if FSHost received the request, and if so, what port it replied to.  I think the best way to get any info from FSHost in this situation is probably through a web page, since that usually works when other things don't.  But how do we send a request on UDP port 23456 and then ask FSHost on TCP 80 (HTTP) whether or not it received it.  The problem is that it's not actually FSHost that's receiving the request -- it's DirectPlay.  And by the time DirectPlay notifies FSHost that it received something, FSHost doesn't know which port it actually came in on (23456 or 6073), or which port the reply went back out on (23456 or 2300-2400).

This last part suggests that instead of using FSHost, we really need a separate test server program, and that it needs to be running on a different IP address from FSHost (or on the same IP when FSHost isn't running).  That way, we could send non-DirectPlay UDP packets from the test client to the test server, and they could communicate with each other on TCP port 80.  So first the client would send a UDP packet on port 23456 saying, "This is test number 42" to the server.  Then the client would make a request on TCP port 80 to the server, and ask, "What have you received lately?".  And the server would send back, "I received 'test number 42' from IP 12.12.12.12 at 12:34:56." (etc.)

This would mean that for most 24/7 servers, they'd have to have a second machine with a separate IP address, where they could run this test server.  Or maybe some kind soul could donate a server for everyone to use (sorry, don't have one, or I'd offer).

Ok, I'm gonna stop talking now, and let you guys come up with all kinds of great ideas that I haven't thought of :-)

Russell

[jordan]

Excellent response.  I think you took it a few steps further than I had envisioned.  There is so little feedback from the sim that almost anything above that would be better.

Being a good tester, I noticed that your post involved always receiving solid confirmation or additional information from the server. Perhaps on a web port.  Or the use of a secondary test server on another IP.  Although that is the only way to have a rock solid test...What if that was avoided?  What about a client that makes FS2002 and FS2004 style requests to the server.  At that point the client should be able to detect A. If there were any problems on the way out.  B.  If there were the normal responses after a given period of time.

1.. You are using DirectX ##.#

User selects Version to test? FS2002/FS2004

2.. Sending FS2004 Test...No Send Problems Detected.
3...Waiting for Responses.....waiting...waiting...No Reponses...Continue waiting?....waiting...waiting....No Responses.

or

2.. Sending FS2002 Test...No Send Problems Detected.
3...Waiting for Responses.....waiting...waiting...No Reponses...Continue waiting?....waiting...waiting....No Responses.


I really dislike it when people simplify complex problems, so apologies in advance if I appear to be doing that here.  I respect the fact that you have taken the time to answer at length.

Russell, how is it that some applications slice through routers/firewalls, proxies, NATs, and ISPs with few problems, and little additional configuration. (Teamspeak comes to mind) where as things like Directplay are killing us with all of these port fowards and triggers that need to be manually configured, and even then sometimes refuse to work.

[Russell Gilbert]

No problem about simplifying the solution -- often that's the best way.

Detecting the version of DirectX is no problem.  And sending a DirectPlay request is no problem either, nor is receiving the reply.  But I think the problem is that if the client doesn't get a reply back (which is usually going to be the case), it's hard to know what happened.  Maybe the request never made it to FSHost, or more likely, it did but the reply didn't make it back.

And since DirectPlay always checks 23456 and 6073, and possibly a third port if you have it set to something else (such as 23455), it's hard to know which of the three requests (23455, 23456, or 6073) FSHost is replying to.  In the case that FSHost is hosting on 23455, it'll reply back to the 23455 request on port 23455, it'll reply back to the 6073 request on a random port in the 2300-2400 range, and it won't receive the 23456 request at all.

I think the only way to know if FSHost received the request is for it to tell us.  And it also needs to tell us how it replied, so we know what's not making it back.

You asked about other apps...  I think part of the problem is that there's so few places for good info on this.  I've seen several web pages telling people to open all kinds of ports that aren't even related to FS.  For example, some tell you to use triggers, but triggers are only necessary if you've got a network with more than one computer using NAT and you're trying to run FS on both of them at the same time.  Otherwise, port forwarding should be enough.

I've updated the help info on FSHost as well.  Take a look:

http://www.chocolatesoftware.com/fshost/?help.html#ports

Let me know what you think about this latest info...

Russell

[jordan]

I read your updated help information.  That's a start.  One of the weaknesses I see is that all of the information here is aimed at helping the FSHost user get his server set up and open to the world properly.  But that is just the start of the problem.  The real volume comes in the hundreds, probably thousands of users that are actually trying to connect.  Each with different network topologies, and hardware choices/configurations.  I know that at Hovercontrol we'll do our best to create a all-in-one help page to get people on the right track, perhaps we can share content in the end so that we don't duplicate eachother's efforts to help users out.

One of our members brought forth a pretty cool utility, that I have yet to investigate or test, but looks very promising.  Have you heard of/seen it before?  I look foward to checking it out tonight when I get home.

http://www.purenetworks.com/products/portmagic/

[Russell Gilbert]

Interesting program.  Looks like it might be pretty handy for people that have a router but don't know how to configure it.  It wasn't able to get very far on my machine since I don't have a router, so I couldn't really test it.  Let me know if you find out more.

I agree that the real problems come from configuring all the client machines, not the servers.  And like you said, there are so many different configurations that it makes it very difficult to create a help page.  I helped a guy the other day that had two routers in serial.

I'm not sure I'm up to building a help page like that on my own, especially since I don't even have one router.  But I'd be happy to help out if you guys attempt such a thing.  I know pretty well how FS and DirectPlay work, just not how to configure all the routers to work with them.  But if you have some of that knowledge, maybe we can work together on it.

Russell

[CowBoyGuy]

Hey Guys.... this sounds like a fun project to work out. If it's done well, mayby Mr. Gates would buy it!

BTW Jordan, Teamspeak works so well with users because it stays on the one port specifed. A user 'connect' request sent out on port 3785 is replied to on port 3785 by the server. The server admin will need to address any firewall forwarding issues. The user's system will automatically allow return traffic when an action was initiated.

Port 80 works this way too, which is why there are so few problems - Users request a web page off of port 80, and the return packets are sent on port 80, no issues. If a server returned packets on any other port, the web browser would not look at them. DirectX, on the other hand, expects packets to return on alternate ports... sometimes, but not always, thus the end user problems.

Russ, from the descriptions you have of the various scenarios, it may well be worth considering adding some packet sniffing steps to the test clients. Perhaps then you could match up the packets and ports. I know from working issues with my system and some clients, if we both run (freeware) packet sniffers, we can catch some of the missing details, or even track back the return data.

The 3rd system acting as a 'clearing house' for tests seems like a great idea too. If a test client and a test server both reported data to the 3rd system for matching, the 3rd system could produce some reports for users based on web query IP's matching sets of 'I sent test 44' from a client and 'I received test 44' from a server program. If you were to run the test server piece 'sniffing' the active FSHost server data, you may be able to capture the actual port data without interferring with FSHost activities.

If you could create an FSTestClient and FSHostSniffer, and define the FSTestClearingHouse location in both apps, the FSTestClearingHouse could be a rather basic web page to accept http queries and store information in a backend Database. Then http queries to the FSTestClearingHouse server could offerup the reporting. (This is a lot like the webpost2 scripts available for Teamspeak servers to register). I wouldn't mind putting something like that on my Linux server for my guests. Bandwidth of all the messages from the FSHostSniffer would remain inside my LAN reducing bandwidth, CPU usage for the database lookup would not be on the FSHost server, and I would control when the sniffer application was enabled.

does that make any sense?

[PeterR]

This is a little tangential but is related to the topic. It's an idea I've had for some time. Since you're talking about a tool that emulates an FS client, if it does come to fruition, what about giving the tool an API?

Why? So that other flight sims that support multiplayer (e.g. X-Plane, FlightGear) could interface to the tool to join an FSHost session. Of course, it would still be restricted to the Windows versions of these other sims but at least it would facilitate a sim-independent multiplayer experience.

Call it FSGhost or something. FSHost + FSGhost = The ultimate multi-sim, multiplayer server solution.  

[Russell Gilbert]

Hi guys,

CBG, I think you're right, the best way to solve these problems is to use a packet sniffer on both the client and server machines.  Then you can watch what ports are being used, and can usually figure out what you need to do with the router.  It's worked just about every time I've tried it this way.  This means that either the test programs need to also do packet sniffing (and then they could send normal game packets, and possibly we wouldn't even need a separate server program because we would use FSHost), or that we'd have separate client and server programs and they'd talk their own "test" language, and tell each other what they were doing.  The first way is better because it doesn't require a seperate server program (which has to be run on a separate machine from FSHost).  But I've never done any packet-sniffing programming before, so I'll have to do some investigation on that.  The second way is easier to program, but harder to use.

Peter, your idea of an API is interesting...  I'd love to build an interface for the other programs, but I think it'd be very difficult to do because of the fact that the server has to know so much about how each sim works.  For example, FSHost knows about particular issues (bugs?) in FS2002 and FS2004, and does things that are very specific to each sim.  I'm not sure it's possible to build a server that would support multiple sims without the server knowing all the gory details about each sim.  What we need is a server that would just pass packets around between the players and not really care what the packets were.  Maybe one day the sim creators will make the sims so they all talk the same generic language, but until then, I think we're stuck with having to write a server that knows all the gory details.

Thanks for the input guys...  it's always welcome.

Russell