Capturing USB Traffic for Fun and Profit
=========================================

Capturing USB traffic is critical to reverse engineering your shiny
new Linux-incompatible toy.  This article documents my experiences
with this process in the Barry project, as well as feedback for additional
methods of USB capture gleaned from the Barry community.


What do these numbers mean?
---------------------------

The first step is to get the USB specifications themselves.  Fortunately,
they are freely available on the internet at http://www.usb.org/ under
the Developers section.

There are two versions of USB, but the important stuff is similar in both
versions.  Chapters 9 and 11 document the format of the various descriptor
structs involved with communicating with the device, and will be important
in decoding some of the data dumps later on.


Talking to the device
---------------------

Programming the USB device itself does not require a kernel driver.  You
can do it from user space with the libusb library.  This library uses
the usbdevfs filesystem under /proc to pass USB messages to the kernel,
and the device.

As USB is a completely host-driven protocol, meaning that the device
itself cannot initiate messages, a simple "make request, wait for
response" style of programming is quite sufficient in the majority
of cases.  Some of the USB capture logs may appear reversed, in that there
is a read before the write.  Don't be too concerned about that.

The stable version of libusb only supports synchronous communication
with USB, which forces you to use a write/read cycle.  Again, this is
sufficient for most cases and is the path you should use when first
starting out.


Capturing: The Windows Way
--------------------------

Your shiny new device probably has some proprietary software, and if you've
played with it, you likely have it installed already on some Windows system.
This is likely the fastest method to start getting captures.

I used the USBsnoop package from:

	http://benoit.papillault.free.fr/usbsnoop/index.php

I was only able to get it to work on a Windows XP Pro system, and as this
was the only method I knew of at the time, I kept trying different versions
of Windows until I found one that worked.  If you have a Windows 2000 or
2002 system, USBsnoop may not work for you, but it is still simple to try.

USBsnoop comes as a simple EXE.  Whenever you wish to make a capture, you
run the program, which installs the capture driver temporarily and presents
you with a list of devices to listen to.  Click the device, click the
Install button, then plug in your device and run the software.  The logs
generally show up in the windows directory as usbsnoop.log.

When you are finished, copy this log somewhere else for safekeeping,
click the Uninstall button, and try deleting the log to start fresh for your
next capture.  Sometimes it requires a reboot to get rid of the log.

These captures are very helpful to see the bulk of the protocol.  In my
experience, USBsnoop can miss some of the very early setup behaviour,
but still does a smashing job capturing the heavy duty areas of the protocol.

Once you have the logs, you can use the convo.awk script in the Barry
src directory, and the translate.cc program to help analyze the data.

Other Windows Tools:
--------------------

There's a list of additional Windows USB sniffer tools at:
	http://wiki.wireshark.org/Tools#USB_capture


Capturing: The Linux Way, Method 1
----------------------------------

Recent versions of the Linux kernel in the 2.6 series provide their own
way of getting to the low level USB behaviour.  In the usbcore driver/module,
there is a switch you can turn on with the following command:

	echo Y > /sys/module/usbcore/parameters/usbfs_snoop

All USB data going through the usbdevfs interface (this includes all data
transferred through libusb) will be logged from the kernel.  This shows up
in dmesg output, and /var/log/kern.log on most systems.

The sheer amount of data that can be generated in this manner can sometimes
overwhelm the dmesg kernel buffer, and some USB messages can be lost.  There
are two adjustments you can make to combat this.

	1) I compile a custom kernel with CONFIG_LOG_BUF_SHIFT=21,
		the largest I can make it.  I also have a custom
		patch that limits the size of the USB capture data.
		This patch for kernel 2.6.26.8 is included in the doc/
		directory, and should be relatively easy to apply to
		other kernel versions.

	2) Some distributions have syslog configured to send kernel messages
		to multiple logfile destinations.  Change this to only
		one location, and disable any setting that forces a sync
		after each log message.  This will boost logging speed, and
		reduce the chances of missing messages.

		Normally, this involves a syslog.conf line like this:
			kern.*               -/var/log/kern.log

		You may only want to disable the sync temporarily, as normally,
		you want to guarantee that important kernel messages get saved.

The data captured is very raw, in disorganized hex.  Use the ktrans
program in the tools/ directory to convert it to something readable.

But what if you only have a Windows driver?  The nice thing is that VMWare
uses the usbdevfs interface to share USB devices with the virtual machines.
So, install windows in a VMWare session, install your proprietary drivers
and software, and watch the logging goodness appear from Linux.

As of December 2006, you can still download a free version of VMWare server
from:

	http://www.vmware.com/download/server/


Capturing: The Linux Way, Method 2
----------------------------------

This method is independent of the usbdevfs driver, to my knowledge, and
is the main method of debugging USB problems under Linux.  It can also be
used to capture traffic that is sent by a virtual machine such as
VirtualBox or VMWare.

Recent versions of the Linux kernel provide a binary interface to the
usbmon logging method.  You can simply cat the /sys/kernel/debug/usb/usbmon/1u
file, depending on your bus number, for a text version of the capture,
but the data is limited to 32 bytes.  This is not sufficient for
many of the bulk data transfers used by the BlackBerry.

Wireshark 1.2.x plus libpcap 1.0 or higher will use the /dev/usbmon*
devices to capture the full binary USB traffic.  Wireshark can also
export this data in plain text format, if the GUI display does not
work for you.

Barry includes a copy of the command line usbmon-6 tool in the source tree.
If you are capturing vast amounts of data, this is the tool to use.

It is not guaranteed that the binary interface will capture all data
if the system is really busy.  If this is a concern, renice your VM
and wireshark appropriately.

For more low level information on the usbmon binary interface, see
the file Documentation/usb/usbmon.txt in the Linux kernel sources.


Capturing: The VMWare way
-------------------------

If you're running VMWare anyway, it has a built-in capability to log USB
traffic.

In addition, the Virtual USB Analyzer project provides a graphical way
of viewing USB traffic, and can be found here:

	http://vusb-analyzer.sourceforge.net/

There is a tutorial there, describing how to configure VMWare to log USB
traffic, and how to make use of those logs with vusb-analyzer.


Capturing: The RIM way
----------------------

It was reported on the mailing list back in 2009 that it is possible to add
the following entries to the Windows Registry to cause RIM's software
to create verbose logs in your temp directory.  Apparently this also
enables sniffing the communication with the BlackBerry simulator.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RimUsb\Parameters]
"FileDump"=dword:00000001
"ServiceDebug"=dword:ffffffff

This feature may have changed in more recent releases of RIM's Windows
software, so your mileage may vary.


Capturing: The expensive hardware way
-------------------------------------

Toby Gray reports:

   If you've got (quite a lot of) money to spend then you could also go
   for a hardware USB analyzer, such as the LeCroy Advisor T3
   (http://www.lecroy.com/protocolanalyzer/protocoloverview.aspx?seriesid=280).


Capturing: The BeagleBoard way
------------------------------

There is also a project that makes use of the BeagleBoard
(http://beagleboard.org/) to make it behave as a proxy, to sniff USB
traffic.  You can find more about it here:
	http://elinux.org/BeagleBoard/GSoC/2010_Projects/USBSniffer


Happy Hacking!

Chris Frey <cdfrey@foursquare.net>

