J2i.Net

Nothing at all and Everything in general.

Fallout 4 Pipboy Traffic Logging in C#

Download Code (2.88 mb)

Bethseda Games released Fallout 4 with an accessory application known as Pip-boy. Within the game the main character wears a computing device on the wrist that gives information relevant to what's going on. Current location, objectives, inventory, status, and other information is available through this device. The Pip-boy application communicates with the game to show the same information on one's mobile device. Being able to have this information accessible while playing the game can be a lot more convenient than the alternative; pausing the game to view the same information. 

If you look around on the Internet you can find a couple of sites that talk about the protocol that the game uses to communicate with the application. It appears that creating one's own client for the game isn't difficult. I've got some other ideas on accessory applications that can be made. Rather than test the application against a live network stream initially I wanted to grab the traffic from the Pip-boy application and save it to a file (this makes debugging much easier).  So I made a quick console application that does just this.  I came across information on the communication between the Pip-boy and the game console in a blog entry by Kyle Kelley named "Fallout 4 Service Discovery and Relay"I'd encourage you to go read the entry to see how he figured this out. 

To summarize what's going on there is communication occurring over two channels. For discovery the game console is listening for UDP broadcast messages on port 28000. A client wishing to discover the game console only need broadcast a message with a small JSON payload to get a response back from the machine that host the game. A relay application will both need to request the address of the real game console and also be prepared to respond to the same type of broadcast request so that a real Pip-boy client can discover it. The relay application also needs to make sure it's not discovering itself. If the relay application starts listening for broadcast before making it's own to find the real game machine then it will receive a copy of it's own request. This can be easily remedied by having the relay application not respond to any discovery requests until after it has already found the game machine. While I tested this and found it to work another potential solution is to include additional information in the JSON message that can be used to recognize request that originated on the relay server. The code for making the relay discoverable to the Pip-boy follows.

 

//prepare to respond to incoming discovery requests
UdpClient discoveryListener = new UdpClient(PIP_AUTODISCOVER);
discoveryListener.EnableBroadcast = true;
discoveryListener.BeginReceive(DiscoverRequestReceived, discoveryListener);

When a Pip-boy client broadcast it's request the DiscoverRequestReceived method runs.

const int PIP_AUTODISCOVER = 28000;

static void DiscoverRequestReceived(IAsyncResult result)
{
    //this request is from a remote client. Send back an acknowledgement
    UdpClient discoveryListener = (UdpClient)result.AsyncState;
    IPEndPoint ep = new IPEndPoint(IPAddress.Any, 0);

    //If the request came from the machine on which this program is hosted it could
    //be from this very same program! This could result in the program connecting
    //with itself if we don't have a strategy for handling it. For my strategy I will
    //just add something extra to the JSON data that this program can recognize. If
    //you'd like to use a different strategy uncomment the following code if you need
    //to recognize that the request came from the same machine. 
    bool requestFromHostMaching = false;
    var hostEntry = Dns.GetHostByName(Dns.GetHostName());
    var matchingAddress = hostEntry.AddressList.Where((x) => x.Equals(ep.Address)).FirstOrDefault();
    if (matchingAddress != null)
        requestFromHostMaching = true;

    //Parse the response and make sure it appears to be valid
    byte[] data = discoveryListener.EndReceive(result, ref ep);
    string dataString = Encoding.UTF8.GetString(data);
    DiscoverRequest response = (DiscoverRequest)JsonConvert.DeserializeObject(dataString, typeof(DiscoverRequest));
    //If the response is invalid do nothing further. 
    //if the request came from this program ignore it
    if ((response == null) || (response.IsPassthrough))
    {

        discoveryListener.BeginReceive(DiscoverRequestReceived, discoveryListener);
        return;
    }

    Console.Out.WriteLineAsync(String.Format("Request payload: {0}", dataString));

    //Note, the Passthrough member is not part of the request that comes from 
    //the game console. I've added it so that I can recognize requests from my
    //own machine. 
    byte[] responseBytes = StringToBytes(JsonConvert.SerializeObject(new DiscoveryResponse{ IsBusy = isBusy, MachineType = "PC", IsPassthrough = true }));
    discoveryListener.Send(responseBytes, responseBytes.Length, ep);
    Console.Out.WriteLine("[{0}]Discover request received from {1}", DateTime.Now, ep.Address);

    //Listen for next request
    discoveryListener.BeginReceive(DiscoverRequestReceived, discoveryListener);
}

public class DiscoverRequest
{
    public DiscoverRequest()
    {
        IsPassthrough = false;
    }
    public string cmd { get; set; }
    public bool IsPassthrough { get; set; }
}
public class DiscoveryResponse
{
    public DiscoveryResponse()
    {
        IsPassthrough = false;
    }
    public bool IsBusy { get; set; }
    public string MachineType { get; set; }
    public bool IsPassthrough { get; set; }

}

Now that the Pip-boy can discover the relay we still need for the relay to discover the real game console.

//discover the real service
UdpClient client = new UdpClient();
var discoverMessage = JsonConvert.SerializeObject(new DiscoverRequest { cmd = "autodiscover", IsPassthrough = true });

byte[] sendbuf = Encoding.ASCII.GetBytes(discoverMessage);
byte[] recvBuffer = new byte[128];
IPEndPoint ep = new IPEndPoint(IPAddress.Broadcast, PIP_AUTODISCOVER);

client.Send(sendbuf, sendbuf.Length, ep);
recvBuffer = client.Receive(ref ep);
var response = Encoding.ASCII.GetString(recvBuffer);
var responseMessage = JsonConvert.DeserializeObject(response);
Console.WriteLine(response);
Console.Out.WriteLine("[{0}] Game instance found at {1}", DateTime.Now, ep.Address);

With discovery handled all that's left is to ferry the bytes back and forth between the Pip-boy and the game console; a copy of the bytes from the game console will also be written to a file. All the data that we care about will be communicated with TCP/IP. The .Net library allows us to interact with a TCP client as a stream; many of the same methods that one might use for interacting with a file stream can be applied here. The transfer of the bytes is simple. Read some number of bytes from a source. Write those same bytes to a destination and if desired write another copy of those bytes to a file stream.

async static void TransferData(TcpClient sourceClient, TcpClient destinationClient,  Stream saveStream = null)
{
    if (isClosing)
        return;
    try
    {
        var sourceStream = sourceClient.GetStream();
        var destinationStream = destinationClient.GetStream();
        if (sourceStream.CanRead && destinationStream.CanWrite)
        {
            byte[] readBuffer = new byte[1024];
            int bytesRead = await sourceStream.ReadAsync(readBuffer, 0, readBuffer.Length);
            if (bytesRead > 0)
            {
                await destinationStream.WriteAsync(readBuffer, 0, bytesRead);
                if (saveStream != null)
                {
                    await saveStream.WriteAsync(readBuffer, 0, bytesRead);
                    await saveStream.FlushAsync();
                }
            }
            TransferData(sourceClient, destinationClient, saveStream);

        }
    }
    catch (IOException)
    {
        Console.Out.WriteLine("Closing Session");
        isClosing = true;
        sourceClient.Close();
        destinationClient.Close();
    }
}

In the last line of this method it appears to call itself. Whether or not this is recursion I'll leave open to argument. This call will never result in stack space being exhausted. With TransferData being marked as asynchronous a call to this method doesn't result in a call that takes execution a level deeper into the stack. Instead it schedules the execution of this method in the thread pool. The optional last parameter to this method is a stream to which a copy of the data is to be saved. For my purposes I only wanted to save data that was coming from the game console. I provide a file stream for one call to the TransferData method and not another.

TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.Any, PIP_CLIENT));
listener.Start();
while (true)
{
    TcpClient remoteClient = listener.AcceptTcpClient();
    isBusy = true;
    ep.Port = PIP_CLIENT;
    TcpClient pipClient = new TcpClient(ep.Address.ToString(), PIP_CLIENT);
    
    FileStream gameMessageCaptureStream = new FileStream("capture.data", FileMode.Create);
    TransferData(pipClient, remoteClient,  gameMessageCaptureStream);
    TransferData(remoteClient, pipClient);

    Console.Out.WriteLine("[{0}] Connection accepted from {1}", DateTime.Now, remoteClient.Client.RemoteEndPoint);

    while (!isClosing)
        Thread.Sleep(100);
    isBusy = true;
    gameMessageCaptureStream.Close();

    Thread.Sleep(50);
    isClosing = false;

}

That's all the code that I used. Now that I've got a saved data stream I've managed to successfully parse all the data that the Pip-boy sends back and forth. I'm still working on understanding it right now.  Once I understand it I hope to make a client in C# for other purposes. 

 

 

What is Listening on my Port!

I wanted to do a quick post on a last minute problem and solution I encountered.

I'm heading out to do a presentation on Azure for Windows Phone developers in about 30 minutes and as I was running through my code I came across a problem; the Azure Emulator was hosting my services on port 444 instead of the HTTPS port (443). It was obvious that something must be using port 443 and that the service has decided to use the next port up (444). Problem is the application is not configured to run on this port and I didn't want to go through and start updating configuration files just before a presentation. So how do I find what is occupying that port?

There's a command line tool called netstat that can be used to answer this question. I'll save you the trouble of looking through it's documentation. The exact command to type in is as follows:

netstat -o -a

It will return a list of addresses, ports, and process IDs. I looked for the line that mentioned port 443 and saw a process ID of 4332. From there I opened the task manager ( [CTRL]+[SHIFT]+[ESC]) and found process 4332 in the "Services" tab. It was a service from VMWare. I shut down the service, stopped the Azure emulator, and restarted the project. This time it came up on port 433!