Sunday, March 1, 2009

RegRipper and Volatility Prototype

When I first released the registry tools for Volatility, I discussed the possibility of interoperating with Harlan Carvey's excellent RegRipper. Now, thanks to Inline::Python and a bit of hackery, you can now run RegRipper against a memory image! Unfortunately, since Inline::Python only seems to work on Linux, you'll need to have a working Linux box around to use this (if anyone knows of a cross-platform way to use Python code from Perl, please let me know!).

I'll get to the details of how this works later, but for now let's talk about how you actually use this stuff. First of all, since we depend on Inline::Python to manage the unholy union of Perl and Python, you'll need to get it from CPAN or your distribution's package manager. No need to install Parse::Win32Registry; I've replaced it with my own registry code that will run against memory.

Next, you should download the latest version of the registry tools [tarball, zip] (side note: I updated them yet again, fixing a couple bugs and adding some basic checking to make sure you've got the right hives when using the credential dumping plugins), as well as the VolRip package [tarball, zip]. Unpack both into your Volatility directory.

Finally, you'll need a list of the virtual addresses of each registry hive in the image you want to examine. You can generate this list by first running the hivescan plugin to find the physical address of any hive, and then passing that to the hivelist plugin. You can see the results for the xp-laptop-2005-07-04 image here. For this example, we'll be using the SYSTEM hive, which is at address 0xe1035b60 in that image.

Now, let's run RegRipper! We're going to run the system plugins against the xp-laptop-2005-07-04 image, like so:

perl rip.pl -r xp-laptop-2005-07-04-1430.img@0xe1035b60 -f system

You can see the results here, if you can't follow along directly at home. If you're familiar with RegRipper, you'll notice that where we usually give the path to the registry hive, we've instead given the path to the memory image, and the virtual address of the hive we want to examine (here, the SYSTEM hive).

That's really about all there is to it! You use rip.pl just as you always would, but instead of giving it the hive filename, you instead pass the image and address of the hive, separated by an "@" sign. You can still run individual plugins using -p, use the "hive guessing" feature with -g, or run a whole batch of plugins using -f. Now, if you want to know more about how this works, read on...

The basic idea was to create a set of Python objects that emulated the interface provided by James Macfarlane's Parse::Win32Registry as closely as possible. This is what regwrap.py does: provides a translation layer between my own registry API and the one exposed by Parse::Win32Registry. Once we've done that, we can drop this emulated object into rip.pl, and the Perl code will run happily, thinking it's operating on normal hive files.

I had to make the following changes to rip.pl to get this to work:
  1. Instead of importing Parse::Win32Registry, use Inline::Python to import my own wrapper class and bind it to the name "Parse::Win32Registry".
  2. Comment out the lines that check to make sure the hive file exists -- since we're smushing the memory image and hive address into a single "filename", those checks will now fail.
  3. Minor changes so that rip.pl works on Linux.
As you can see, not many changes were required. With just these modifications, most RegRipper plugins run perfectly.

However, there was still one small wart that I couldn't get rid of just by emulating the library. See, in Perl, you can call a function in two contexts: "scalar" and "array". This basically means, "is the function result being stored in an array, or in a scalar variable"--and functions can decide to do different things depending on what context they're called from. An example of this is the get_data method of Value objects in Parse::Win32Registry.

The get_data method, when called on a REG_MULTI_SZ value, will return an array of strings if called from array context, but returns a string with all the strings separated by spaces if called in a scalar context. Unfortuantely, the Python code has no way of telling what Perl context it's being called from, so my wrapper always returns a list. This means that a couple of plugins (shares.pl and nic_mst2.pl) would print "ARRAY(0x12345)" instead of the actual data in some cases.

Instead of trying to come up with a convoluted way around this, I opted to just change the two plugins in question to remove any ambiguity as to how the function was called. I'm still not sure I've identified all the plugins that do this (I tried to run them all to check, but may have missed something), but the results look good to me now.

So, to summarize: it's now possible to run RegRipper against memory images! The modifications required were pretty minimal, so it should be easy to keep things updated as new versions of RegRipper come out. In particular, aside from the two plugins I mentioned, upgrading the plugins should be as easy as just dropping them into the rrplugins directory.

Enjoy, and as always, let me know if you find any bugs :)

21 comments:

Keydet89 said...

Very, VERY cool!

chris said...

I'm gettin an error with hashdump

cg@attack:~/evil/msf3/scripts/Volatility-1.3_Beta$ python volatility hivelist -f demo.dd -o 0x8f8b60

0xe13ef638 \WINDOWS\system32\config\SAM
0xe13d8b60 \WINDOWS\system32\config\default
0xe13d8488 \WINDOWS\system32\config\SECURITY
0xe12cca80 [no name]
0xe101b008 \WINDOWS\system32\config\system


cg@attack:~/evil/msf3/scripts/Volatility-1.3_Beta$ python volatility hashdump -f demo.dd -y 0xe101b008 -s 0xe13ef638
Traceback (most recent call last):
File "volatility", line 219, in module
main()
File "volatility", line 215, in main
command.execute()
File "memory_plugins/registry/hashdump.py", line 72, in execute
if not samhiveobj.Hive.BaseBlock.FileName.endswith("SAM"):
File "/home/cg/evil/msf3/scripts/Volatility-1.3_Beta/forensics/object2.py", line 144, in __getattribute__
array_vals.append(element.v())
File "/home/cg/evil/msf3/scripts/Volatility-1.3_Beta/forensics/object2.py", line 251, in v
return self.value()
File "/home/cg/evil/msf3/scripts/Volatility-1.3_Beta/forensics/object2.py", line 255, in value
return self.type.v(self)
File "/home/cg/evil/msf3/scripts/Volatility-1.3_Beta/forensics/object2.py", line 340, in v
return self.value(theObject)
File "/home/cg/evil/msf3/scripts/Volatility-1.3_Beta/forensics/object2.py", line 344, in value
theObject.vm.read(theObject.offset, self.size))
File "/usr/lib/python2.5/struct.py", line 87, in unpack
return o.unpack(s)
struct.error: unpack requires a string argument of length 1

any ideas? I took this from a live VM XP Pro SP1

chris said...

yeah RTFM, XP SP2 & SP3 only.

moyix said...

chris,

I'm not sure your error is related SP1. Is there any way you could provide the ntoskrnl.exe from that machine?

I've also added some more error checking around that area of code in the version of VolReg I just released. Could you try it out and see if it fixes your problem?

Thanks,
Brendan

chris said...

Thanks for the reply.

I'll try the updated code, if that doesnt fix it i'll send you ntoskrnl.exe.

Chris

Howard said...

When I try the hivescan (which now seems to be hivescan2) command it errors with "Invalid module [hivescan2]". I am using it on an XP SP3 memory image.

moyix said...

Howard,

I'm not sure what you mean -- the hivescan plugin has not been renamed. Running

python volatility hivescan -f [image]

should give you a list of offsets to hives in the memory image.

Howard said...

In the volreg-0.3.zip I downloaded, the file in the archive is "hivescan2.py".

moyix said...

Unfortunately the filename you see in the archive does not match up with the plugin name (as seen by Volatility). I'll fix this in a future release. "hivescan" is the correct command to use. Is this working for you?

Also, note that the current version of VolReg is 0.4. You can always find the most recent version of my plugins at:

http://www.cc.gatech.edu/~brendan/volatility/

Howard said...

After reinstalling the Volatility beta and the modules, I got it to work. Thanks.

johnmccash said...

I finally set up a Linux VM to run this in, and I'm having some problems with it. I first tried it in a CentOS VM, but CentOS is dependant on an old version of python, and I assumed my problems werre a result of the old and new versions clashing somehow. But then I tried it in the new Fedora 10, and I get the same or very similar issues. Here's the output:

Parsed Plugins file.
Launching compname v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in compname: PyObject_CallObject(...) failed.

compname complete.
Launching shutdown v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in shutdown: PyObject_CallObject(...) failed.

shutdown complete.
Launching shutdowncount v.20080709
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in shutdowncount: PyObject_CallObject(...) failed.

shutdowncount complete.
Launching timezone v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in timezone: PyObject_CallObject(...) failed.

timezone complete.
Launching termserv v.20080418
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in termserv: PyObject_CallObject(...) failed.

termserv complete.
Launching mountdev v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in mountdev: PyObject_CallObject(...) failed.

mountdev complete.
Launching network v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in network: PyObject_CallObject(...) failed.

network complete.
Launching nic_mst2 v.20080324
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in nic_mst2: PyObject_CallObject(...) failed.

nic_mst2 complete.
Launching fw_config v.20080328
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in fw_config: PyObject_CallObject(...) failed.

fw_config complete.
Launching usbstor v.20080418
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in usbstor: PyObject_CallObject(...) failed.

usbstor complete.
Launching devclass v.20080331
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in devclass: PyObject_CallObject(...) failed.

devclass complete.
Launching ide v.20080418
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in ide: PyObject_CallObject(...) failed.

ide complete.
Launching shares v.200800420
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
shares complete.
Launching svc v.20080610
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
Error in svc: PyObject_CallObject(...) failed.

svc complete.
Launching imagedev v.20080730
Traceback (most recent call last):
File "regwrap.py", line 96, in get_subkey
volsk = open_key(self.volkey, keyname_l)
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 95, in open_key
for s in subkeys(root):
File "/home/x/Desktop/Volatility/forensics/win32/rawreg.py", line 142, in subkeys
s.is_valid() and s.Signature == NK_SIG]
AttributeError: 'int' object has no attribute 'is_valid'
imagedev complete.
----------------------------------------
----------------------------------------
----------------------------------------
----------------------------------------
----------------------------------------
mountdev v.20080324
Get MountedDevices key information from the System hive file.

----------------------------------------
----------------------------------------
----------------------------------------
----------------------------------------
----------------------------------------
----------------------------------------
IDE
----------------------------------------
Problem locating proper controlset: PyObject_CallObject(...) failed.

----------------------------------------
----------------------------------------
Problem locating proper controlset: PyObject_CallObject(...) failed.

----------------------------------------

moyix said...

Hi John,

I think the issue is with a change that was recently introduced to the Volatility SVN repostiory. We've backed out the change now; can you try the most recent SVN and let me know if it's still giving you trouble?

Thanks,
Brendan

johnmccash said...

It looks like it works OK now. Anything I should watch out for? I assume the change that was backed out was put in for a reason...

johnmccash said...

Guys, I was just re-testing all the commands after the upgrade, and I saw something I noticed but blew off before. Are thrdscan and thrdscan2 supposed to produce the same output? Thrdscan2 seems to leave off the first column of data, though the heading is the same...

moyix said...

The change that was put in was to prevent an error that crops up when you have two plugins that both define an object with the same name. This was happening with one of Jesse Kornblum's plugins because he based it off one of the example plugins (which defined a particular object).

Anyways, we're working on a fix for it that doesn't break existing plugins...

As for the thrdscan question, you are correct -- thrdscan2 is not printing out the "No." field. Most likely the header for the old one got copied over into the new one... in any case, the content of the results should be the same for both.

johnmccash said...

Ahah! That explains why I had to delete that example plugin to get Jesse's to work. I didn't think I'd had to do that before, but had chalked it up to general confusion.

Michael Hale Ligh said...

Hey BDG, just thought I'd drop you a note and say that Volreg works great on an OSX system. Also I think that last comment by "Shown" is spam ;-)

moyix said...

Cool! I figured Volreg would work fine on OSX, since I developed it ther ;)

Unfortunately, the Perl integration is going to be more of a problem, since Inline::Python is currently Linux-only.

Michael Hale Ligh said...

The VolRip Perl integration works fine on OSX too (I actually meant to type "VolRip" instead of "VolReg" in my original comment). Looks like they updated Inline::Python to 0.31 in Dec 2009.

moyix said...

Sweet! I wonder if that update made things work with Windows, too...

peppespe said...

I premise that I work on a Mac OS X 10.7 (Lion).
I tryed to use volrip-0.1, but I received this message error:

MacBook-di-peppespe:volrip-0.1 giuseppespecchio$ perl rip.pl -r /Users/giuseppespecchio/Desktop/Volatility/RAMDump/xp-laptop-2005-07-04-1430.img 0x0283db60 -f system
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named vtypes
Error -- py_eval raised an exception at rip.pl line 21.

where line 21 is this:

py_eval(<<'END');

Similarly I tryed to use RegRipper032911, but i received this message error:

$ perl rip.pl -r /Users/giuseppespecchio/Desktop/Volatility/RAMDump/xp-laptop-2005-07-04-1430.img 0x0283db60 -f system
Parsed Plugins file.
Launching auditfail v.20081212
Error in auditfail: Can't call method "get_root_key" on an undefined value at plugins/auditfail.pl line 40.

auditfail complete.
----------------------------------------
Launching compname v.20090727
Error in compname: Can't call method "get_root_key" on an undefined value at plugins/compname.pl line 40.

compname complete.
----------------------------------------
Launching crashcontrol v.20081212
Error in crashcontrol: Can't call method "get_root_key" on an undefined value at plugins/crashcontrol.pl line 41.

crashcontrol complete.
----------------------------------------
<>
----------------------------------------
Launching xpedition v.20090727
Error in xpedition: Can't call method "get_root_key" on an undefined value at plugins/xpedition.pl line 42.

xpedition complete.
----------------------------------------