SedTris - a Tetris game written in sed.

I've seen irc clients written in sed and I've seen a lot of sed magic in my time, but this is just ridiculous...

A Tetris game written in sed.

http://uuner.doslash.org/forfun/sedtris.sed

AWK tricks - clearing the screen and scrollback buffer.

You can put this in your logout profile to fill up the screen buffer (clear just won't do on some systems with a multiple page console scroll-back buffer).

Filling the screen buffer with new lines:

awk -F 'BEGIN{ for (i = 1; i <= 3000; i++) printf "\n"; exit}'

See also:

http://www.openbsd.org/faq/faq7.html#ConsoleClear

Using AWK to convert UNIX passwords from HP-UX to Solaris

Converting password hashes from HP-UX 11.11 to Solaris is pretty simple if you are using UNIX crypt passwords (if HP-UX isn't a Trusted System. If it is, it will use bigcrypt passwords, > 8 characters, converting them to Solaris UNIX crypt could be problematic).

Here's the gest of it:

On the HP-UX System, we create a test user:

# useradd test
# passwd test
test

Now we convert the passwd file to generate passwd entries for Solaris:

• # awk ' BEGIN { FS = ":" } { print $1":x:" $3 ":" $4 "::/export/home/" $1 ":/usr/bin/sh" }' /etc/passwd
• test:x:107:20::/export/home/test:/usr/bin/sh

And we create the shadow file entries, including the password hash:

• # awk ' BEGIN { FS = ":" } { print $1":"$2"::::::" }' /etc/passwd
• test:lsDWJo7M.iAhY::::::

Just add them using /usr/ucb/vipw to the password file, edit the shadow file for consistency and test. Be sure to backup the files and to test using a few users at first.

• $ su test
• Password:
• $ id
• uid=127(test) gid=120
• $ whoami
• test
• $ echo $HOME
• /export/home/test
• $ echo $SHELL
• /usr/bin/sh

Mix with some shell scripting and mkdir's and you're set :-). Next time, use LDAP :P.

Custom df (diskfree) column output in Solaris using nawk

Let's say you want to combine some features of "df -h" with "df -n" to show filesystem type and some other custom modifications to the output. This is where awk/nakw/gawk/whatever come in handy:

% df -g | nawk '{if (NR % 5 == 1) printf "%-22s", $1 ; if (NR % 5 == 4) printf "%-10s", "fstype " $1 "\n"; if (NR % 5 == 2) printf "%-30s",$1/2/1024/1024 " GB"; if (NR % 5 == 2) printf "%-30s", $4/2/1024/1024 " GB free "}'

/	33.6627 GB	18.4351 GB free	fstype ufs
/devices	0 GB	0 GB free	fstype devfs
/system/contract	0 GB	0 GB free	fstype ctfs
/proc	0 GB	0 GB free	fstype proc
/etc/mnttab	0 GB	0 GB free	fstype mntfs
/etc/svc/volatile	7.88214 GB	7.8813 GB free	fstype tmpfs
/system/object	0 GB	0 GB free	fstype objfs
/lib/libc.so.1	33.6627 GB	18.4351 GB free	fstype ufs
/dev/fd	0 GB	0 GB free	fstype fd
/tmp	7.88142 GB	7.8813 GB free	fstype tmpfs
/var/run	7.88134 GB	7.8813 GB free	fstype tmpfs
/export/home	74.4858 GB	1.87458 GB free	fstype ufs
/storage	108.639 GB	66.9259 GB free	fstype nfs

You can also add a comma (,) to the separators and output > csv (you can open the comma separated values table in Excel or OpenOffice or any other Spreadsheet application) :-).

Convert NRG images to ISO images (nrg2iso) using dd

If you want to convert Nero "nrg" images to ISO format to mount or burn you can either use nrg2iso, or just use dd to skip the 300k header (will work just fine for DAO images):

dd if=image.nrg of=image.iso bs=4096 skip=75

OpenVMS Tips and Tricks - Clear Screen, ls -lha

Want to clear the screen? "TYPE/PAGE NLA0:" will do the trick. Just alias it to your dos like cls or to a UNIX like clear command :-).

$ CLEAR == "TYPE/PAGE NLA0:"
$ CLEAR
$

Might as well alias LS to a DIR/SIZE/DATE/PROTECTION:

$ LS == "DIRECTORY/SIZE/DATE/PROTECTION"
$ LS *.JAVA

Directory SYS$SYSROOT:[SYSMGR]

HELLO.JAVA;1 1 4-MAR-2008 21:40:06.79 (RWED,RWED,RE,)
HW.JAVA;2 1 4-MAR-2008 22:14:36.28 (RWED,RWED,RE,)

Total of 2 files, 2 blocks.

PowerShell, PowerTab, PSCX, Console, Terminus and Vim

Here's something for all you Powershell / WMIC / VB / cmd script monkeys :-).

You can, of course, script in ksh, Perl or Python if that's your thing. (ActiveState Perl / Python).

Powershell + PowerTab + PSCX + Console + Terminus Programmers Fonts + VIM + VIM PowerShell Syntax Highlighting.



Here's another PowerTab Theme and PSCX (PowerShell Community Extensions):



A cyan BackColor to match my prompt:



And of course, a good color scheme and PoSH syntax highlighting in VIM is also nice:

Emacs.NET

"Imagine if someone wanted to write something like Emacs.Net. Actually don't imagine it, it is happening."

"We are looking for developers/testers to build a tool that I will roughly describe as "Emacs.Net"."

^^ duglasp at microsoft dot com

Well, this should be interesting to watch. Hell, notepad could use a rewrite :P.

OpenBSD automated / unattended installation - Yaifo

Don't have remote servers with KVM? Well, this will help:

Yaifo has been updated for OpenBSD 4.2 and is now actively maintained by Mike Erdely. You can download it here:
http://erdelynet.com/category/tech/yaifo/

It's basically a modified installer with ssh support, RAID support and all that. You can basically do remote, scripted, unattended OpenBSD installations, etc.

Uninstalling Blastwave (or any other package group) with a bit of Shell Scripting on Solaris.

Let's say we've installed a bunch of Solaris packages from Blastwave or another source and want to remove them all without going through the boring process of answering all those questions:

$ pkginfo | grep CSW
system CSWcommon common - common files and dirs for CSW packages
application CSWexpat expat - XML Parser Toolkit
system CSWggettext ggettext - GNU gettext
application CSWglib2 glib2 - the low-level core library for GTK+ and GNOME
system CSWiconv libiconv - GNU iconv library
application CSWmc mc - GNU Midnight Commander
system CSWpkgget pkg_get - CSW version of automated package download tool
system CSWslang slang - library primarily to provide an alternative to libcurses
system CSWsunmath libsunmath - Sun maths library

Well, nothing simpler:

# yes | pkgrm $(pkginfo | grep CSW | awk '{ print $2 }' | xargs)

The following package is currently installed:
CSWcommon common - common files and dirs for CSW packages
(i386) 1.4.5

Do you want to remove this package? [y,n,?,q]
## Removing installed package instance
## Verifying package dependencies in global zone

[.....]

Removal of was successful.
# pkginfo | grep CSW
#

Since Blastwave installs by default in /opt/csw, you can probably remove that too if you don't have settings you plan on keeping:

# rm -rf /opt/csw

Enjoy :-).

Killing defunct (zombie) processes on Solaris - preap and orphan processes with init(1M)

A defunct (or zombie) process is one whose exit status has yet to be reaped by its parent. So when a process shows as "defunct" in ps, you need to reap it. Here's how:

preap(1) - force a defunct process to be reaped by its parent

Syntax: /usr/bin/preap PID

So, to get rid of all zombies on our system, all we have to do is script this to real all process marked as defunct:

/usr/bin/preap $(ps -ef | grep defunct | awk '{ print $2 }' | xargs)

So, what's an orphan process then? If the parent of a defunct process happens to exit, it becomes an orphan, and is asigned PID 1 as a parent - see init(1M).

Read more on zombies, defunct processes, orphans, preap and init here.

Debian Etch AMD64 NVIDIA install one-liner

Add contrib and non-free repositories to sources.list:

cmihai@db:~$ cat /etc/apt/sources.list

deb http://ftp.lug.ro/debian/ etch main contrib non-free
deb-src http://ftp.lug.ro/debian/ etch main contrib non-free

deb http://ftp.lug.ro/debian-security etch/updates main contrib non-free
deb-src http://ftp.lug.ro/debian-security etch/updates main contrib n

Install the packages and configure X (nvidia-xconfig).

$ sudo aptitude install nvidia-kernel-2.6-`uname -r | sed 's,.*-,,g'` nvidia-settings nvidia-glx nvidia-xconfig nvidia-glx-ia32 && nvidia-xconfig

PCA - Analyze, download and install patches for Solaris

Trouble with Sun Solaris Patching? Sick of all those tools (smpatch, getsolpatch, Sun Update Connection, Patch Manager PatchPro, PatchCheck, PatchDiag, patchadd) that get dropped or are poorly maintained, broken, etc? Well, here's yet another tool to "fix'em all".

"Patch Check Advanced (pca) generates lists of installed and missing patches for Sun Solaris systems and optionally downloads patches. It resolves dependencies between patches and installs them in correct order. It can be the only tool you ever need for patch management on a single machine or a complete network. Just one perl script, it doesn't need compilation nor installation, and it doesn't need root permissions to run. It works on all versions of Solaris, both SPARC and x86."

Seriously though, PCA rocks ;-).

Cracking Cisco type 7 and type 5 PIX passwords with Cain and Abel

Number one reason you shouldn't paste your Cisco configs or password hashes on the Internet:

Cisco's PIX password encryption is a base64 encoded MD5 hashsum, using only one MD5 update (no salting or anything). This also permits for cryptanalysis attacks using rainbow tables to speed up the process.

Simple hashes like:

enable password RLPMUQ26KL4blgFN encrypted

Get cracked instantly. -> 1234


Also, note that MD5 has know weaknesses in the algorithm, that may allow for more complex password cracking attacks.

Also, if you're using a "type 7" password, that's pretty much useless, since it can get cracked instantly. People can just use simple tools such as Cain and Abel, or Solarwinds Router Password Decryption to reverse the crypto on type 7 passwords.

someuser privilege 0 password 7 06351A3149085123301517391C501918

IOS type 5 passwords (MD5 using 1000 rounds) is more complex, and harder to crack, but yous still shouldn't paste your hashes for everyone to see... Remember, even with type 5 passwords, you're still vulnerable to dictionary attacks, hybrid attacks, rainbow table attacks (PIX only), md5 repository attacks, or plain old password guessing.

If you're going to paste your config files anywhere, use the "show tech-support" command available in newer IOS versions. It gives more info, and strips confidential information, password hashes and such.


You can even use decrypt.pl - a neat little Perl script to instantly decrypt type 7 passwords:

#!/usr/bin/perl -w
# $Id: ios7decrypt.pl,v 1.1 1998/01/11 21:31:12 mesrik Exp $
#
# Credits for orginal code and description hobbit@avian.org,
# SPHiXe, .mudge et al. and for John Bashinski
# for Cisco IOS password encryption facts.
#
# Use for any malice or illegal purposes strictly prohibited!
#

@xlat = ( 0x64, 0x73, 0x66, 0x64, 0x3b, 0x6b, 0x66, 0x6f, 0x41,
0x2c, 0x2e, 0x69, 0x79, 0x65, 0x77, 0x72, 0x6b, 0x6c,
0x64, 0x4a, 0x4b, 0x44, 0x48, 0x53 , 0x55, 0x42 );

}

while (<>) {
if (/(password|md5)\s+7\s+([\da-f]+)/io) {
if (!(length($2) & 1)) {
$ep = $2; $dp = "";
($s, $e) = ($2 =~ /^(..)(.+)/o);
for ($i = 0; $i < length($e); $i+=2) {
$dp .= sprintf "%c",hex(substr($e,$i,2))^$xlat[$s++];
}
s/7\s+$ep/$dp/;
}
}
print;
}

Digital Forensic Tools: Imaging, Virtualization, Cryptanalysis, Steganalysis, Data Recovery, Data Carving, Reverse Engineering

"Jrypbzr gb gur bgure fvqr."

Computer Forensics is a science and an art. And to perform it, you need tools to identify, acquisition, preserve and analyze data in a clean, safe, non-destructive manner. Lots of tools. Everything from data acquisition to virtualization and steganalysis.

A list of more or less free tools (mostly open source or freeware, but I have included some relevant commercial products) no digital forensics expert should be without:

Data acquisition, enumeration, imaging and forensics tools: Toolkits and utilities.

1. The Sleuth Kit and Autopsy Browser. Both are open source digital investigation tools (a.k.a digital forensic tools) that run on Unix systems (such as Linux, OS X, FreeBSD, OpenBSD, and Solaris). They can be used to analyze NTFS, FAT, Ext2, Ext3, UFS1, and UFS2 file systems and several volume system types. The Sleuth Kit (TSK) is a collection of command line tools based on The Coroner's Toolkit (TCT). Autopsy is a graphical interface to TSK.
2. The Coroner's Toolkit (TCT) is a forensics toolkit for analysis of UNIX break-ins. It runs on BSD (OpenBSD, FreeBSD, BSD/OS), Solaris/SunOS, Linux and HP-UX.
   
3. WinHex: Computer Forensics & Data Recovery Software, Hex Editor & Disk Editor.
4. dcfldd is an enhanced version of GNU dd with features useful for forensics and security. GNU ddrescue is a data recovery tool. It copies data from one file or block device trying to rescue data in case of read errors. It's a better alternative to using dd_rescue and dd_rhelp or SpinRite (you can just do a disk refresh with dd: "dd if=/dev/disk of=/dev/disk bs=2m" while the drive isn't mounted - no write operations going on - or something along those lines in order to prevent presently recoverable read errors from progressing into unrecoverable read errors).
5. Sysinternals tools contains programs like streams that help us find data hidden inside alternate streams or strings that grep readable strings from a file. It also has tools like process explorer, procmon, autoruns and rootkit revealer that allow you to dig deep into the Windows operating system to process, disk and data related information.
   
6. Microsoft Log Parser is a powerful, versatile tool that you can use to extract information from files of almost any format by using Structured Query Language to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory.
7. AccessData Forensic Products: FTK - Forensic Toolkit, Registry Viewier - more neat tools from AccessData. Commercial products.
8. Clonezilla is used to clone many computers simultaneously. It can perform a full disk image or just file backup. It's a backup tool, but it can also perform bit by bit disk imaging.
9. Sysinternals LiveKD is a Live version of Windows Debugger (WinDBG) that allows you to run the Kd and Windbg Microsoft kernel debuggers, which are part of the Debugging Tools for Windows package, locally on a live system. This means that you to easily take a memory dump of a running Windows system (.dump /f YOURUSBDISK:\fullmemorydump.dmp). On UNIX systems you can use dd to take a snapshot of the system memory ("dd if=/dev/kmem of=/path/to/memorydump").
10. Paraben's Device Seizure - Cell Phone and PDA Forensic software. Specialized software for portable device forensics.
11. PDD is a forensic analysis tool for Palm OS platform devices. It is an open source Windows-based tool for Palm OS memory imaging and forensic acquisition. The Palm OS Console Mode is used to acquire memory card information and to create a bit-for-bit image of the selected memory region. No data is modified on the target device and the data retrieval is not detectable by the user of the PDA.
12. CDInfo is an application that will display all ISO descriptors from all attached cd-rom drives (Label, System, Application, VolumeSet, Copyright, Creation Date, Directory Start, Directory Length, extentions, tracks, etc).
13. PMDump is a Windows tool that lets you dump the memory contents (both RAM and swap) of a process to a file without stopping the process.

Virtualization: Once the actual machine is cloned, it's usually put inside a virtual machine (features like snapshots and debugging help quite a bit with the digital forensics process). This is called physical to virtual (P2V) migration.

1. Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to "boot up" the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra "throw away" copies of the disk or image to create the virtual machine.
2. VMware Converter Starter is a free p2v (physical to virtual) migration tool. VMware Converter quickly converts Microsoft Windows based physical machines and third party image formats to VMware virtual machines. It also converts virtual machines between VMware platforms. Note: for digital forensic images, you should use LiveView, and not the converter.
   
3. VMware Server allows for free virtualization. You can use it in combination with Live View to virtualize existing environments, and use the snapshots feature to revert back to a previous state of a virtual machine in an instant.
4. QEmu - a much more flexible virtualization program, albeit a bit slower than VMware. Supports emulating IA-32 (x86) PCs, AMD64 PCs, MIPS R4000, Sun's SPARC sun4m, Sun's SPARC sun4u, ARM development boards (Integrator/CP and Versatile/PB), SH4 SHIX board, PowerPC (PReP and Power Macintosh), and ETRAX CRIS architectures. Also, qemu-img can be a valuable tool for converting virtual machine images. Also allows for some really low level debugging features. A modified version of QEmu can even emulate PIX platforms (or Juniper JunOS systems like Olive).
   
5. VirtualBox is a GPL licensed x86 virtualization platform that runs on Windows, Linux and MacOS hosts, and supports various x86 client machines (Windows, Linux, BSD, Solaris). It's a noteworthy alternative to using VMware, as performance tends to be pretty good.
6. Microsoft VirtualServer / Virtual PC are free virtualization products from Microsoft. They support all major features (snapshots, mounting ISO images and such), and performance tends to be reasonable (to some extent, similar to that of VMware).
7. SIMH is a highly portable, multi-system simulator. It can emulate VAX and PDP-11 platforms. Just in case you need to perform forensics on older minicomputers.
8. Hercules is an open source (QPL licensed) emulator of IBM Mainframe computers (System/370, ESA/390 architectures and even the 64-bit zSeries). Hercules runs under Linux, Windows (98, NT, 2000, and XP), FreeBSD, and Mac OS X (10.3 and later). Hercules will run OS/360, DOS/360, DOS/VS, MVS, VM/370, TSS/370 - all IBM public domain operating system, as well as OS/390, z/OS, VSE/ESA, z/VSE, VM/ESA, and z/VM, and even Linux/390 and Linux (SuSE, RHEL, Debian, CentOS and Slackware) on zSeries.
9. Oracle VM is a server virtualization software based on Xen and Oracle Linux (itself based on RHEL sources) that fully supports both Oracle and non-Oracle applications. It is a free alternative to VMware Virtual Infrastructure (VMware ESX + VirtualCenter). It is certified to run the Linux operating system, Oracle Database, Fusion Middleware, and Application software, thus makes a very good platform for investigating Oracle databases.
10. The Palm OS Emulator is a program based on the Copilot app that emulates the hardware of the various models of Palm-powered handhelds, making it a valuable tool for writing, testing, and debugging applications as well as obtainiwinng evidence from the device.
11. Microsoft Windows CE 5.0 Device Emulator contains the emulator technologies featured in Windows CE 5.0. By using the Device Emulator, you can run emulated-based images created by Windows CE 5.0 without installing Platform Builder, its platform development tool.
    

Password recovery tools: You may often need to recover keys and passwords.

"This text has been encrypted twice... for double protection!"

1. Ophcrack is a very efficient Windows password cracker based on rainbow tables. It will crack huge tables of LM hashes in under 3 minutes. It also comes in the form of a LiveCD (though in digital forensics cases it's usually best to extract the SAM file containing the password hashes from the disk image and use that. Ophcrack can be a lot more effective if you have more complete rainbow tables.
2. LCP is a free alternative to the now dead L0phtcrack.
3. John the Ripper is a very versatile password cracking tool. It's supported on different architectures and operating systems (UNIX, Windows, OpenVMS, etc) and it's quite fast. Besides several crypt(3) password hash types most commonly found on various Unix flavors, supported out of the box are Kerberos AFS and Windows NT/2000/XP/2003 LM hashes, plus several more with contributed patches.
4. Medusa is a very fast parallel brute force login password cracker.
5. Elcomsoft Password Recovery suite: anything from office, archives, pdf files etc. to email clients. These are commercial products though.
6. CmosPwd decrypts password stored in CMOS used to access BIOS SETUP. Works on a lot of BIOSes (AMI, Award, Phoenix, IBM, etc). It can also be used to backup, restore or erase the BIOS.
7. AccessData Decryption Tools: PRTK - Password Recovery Toolkit, DNA - Distributed Network Attack, PORT - Portable Office Rainbow Tables are some of the best and fastest tools in the business.
8. Offline NT Password and Registry editor - a utility to (re)set the password of any user that has a valid (local) account on your NT system. You do not need to know the old password to set a new one. Features a registry editor. Supports 32 and 64 versions of Vista (and NTFSv5).
9. Elcomsoft Distributed Password Recovery is designed for distributed recovery of forgotten or lost passwords of different documents. Version 2.0 adds support for Windows SYSKEY startup passwords, passwords stored in Domain Cached Credentials, includes updated Adobe Acrobat module, and provides hardware acceleration (now up to 25 times faster!) for NTLM password recovery using GeForce 8 video cards.
10. Dialupass - Dialup Password Recovery - Recovers the passwords of dialup entries (VPN and Internet connections) on Windows systems. NirSoft provides a couple of free password recovery tools for various products such as Instant Messaging applications, cached passwords stored by Internet Browsers, E-mail clients and so on.

Here's a little cool trick for recovering cached passwords (asterisk passwords) stored in your Internet Browser (Firefox, Opera, Internet Explorer or anything with JavaScript).

Steganalysis and stenography: how to detect hidden data using stenography.

1. Stegdetect finds hidden information in JPEG images using such steganography schemes as F5, Invisible Secrets, JPHide, and JSteg. XSteg is a GUI.
2. Stego Suite is a powerful commercial Stenography detection toolkit, consisting of 3 major tools.
3. Stegkit is an Automated Steganalysis Tool.
4. Digital Invisible Ink Toolkit is an open-source cross-platform image steganography suite that includes both steganography and steganalysis implementations.
5. StegSpy will detect steganography and the program used to hide the message.
   
6. SteGUI is a StegHide GUI.
7. Digital Watermarking allows you to hide copywrite information and such in media (images and such) that's present even after encoding to another format (bmp->jpg), printing, copy/paste, etc. You can use ImageMagick of various Photoshop plugins to do this.
8. Stepic is a Python module and command line tool for hiding arbitrary data within images by slightly modifying the colors. These modifications are generally imperceptible to humans, but are machine detectable.
9. wbStego4 offers steganography in bitmaps, text files, HTML files and PDF files. It is has two very user-friendly interfaces and is ideal for securely transmitting data online or adding copyright information, especially with the copyright information manager.
   
10. NL Stego is a system for text generation and text-based steganography. It combines Markov Models of several orders to generate random text resembling a given training text (or text corpus). It can also embed secret messages into pseudo-random generated text.
11. Steghide is an Open Source (GPL) steganography program that is able to hide data in various kinds of image- and audio-files. The color- respectivly sample-frequencies are not changed thus making the embedding resistant against first-order statistical tests. Supports compression of embedded data, encryption of embedded data, embedding of a checksum to verify the integrity of the extraced data and has support for JPEG, BMP, WAV and AU files.
    
12. StegFS is an Open Source (GPL) Steganographic File System for Linux. Not only does it encrypt data, it also hides it such that it cannot be proved to be there.
    

Filesystem tools: Data Recovery.

1. Undelete Plus is a free deleted file recovery tool that works for all versions of Windows (95-Vista), FAT12/16/32, NTFS and NTFS5 filesystems and can perform recovery on various solid state devices.
2. Avira UnErase Personal - a freely available unerase product.
   
3. TestDisk is a free (GPL) data recovery software that can fix partition tables, recover deleted partitions and rebuild NTFS boot sectors. It can find lost partitions (anything from BSD disklabels to IBM JFS, it supports pretty much anything).
   
4. GNU Parted is a program for creating, destroying, resizing, checking, and copying partitions, and the file systems on them. This is useful for creating space for new operating systems, reorganising hard disk usage, copying data between hard disks, and disk imaging. It can also be used to attempt recovery of the partition table similar to TestDisk (rescue START END).
   
5. Stellar Phoenix has various UNIX and *NIX (SCO OpenServer, Unixware, Sun Solaris, *BSD, HP-UX, MacOS) data recovery tools as well as some Windows Data Recovery tools. They are, however, commercial products.
6. R-Studio is a data recovery software suite that can recover files from FAT(12-32), NTFS, NTFS 5, HFS/HFS+, FFS, UFS/UFS2 (*BSD, Solaris), Ext2/Ext3 (Linux) and so on. This is a commercial product.
7. DiskInternals NTFS Recovery is a fully automatic utility that recovers data from damaged or formatted disks. This is a commercial product.
8. SystemRescueCD is a Linux system on a bootable CD/DVD for repairing your system and your data after a crash. It also aims to provide an easy way to carry out admin tasks on your computer, such as creating and editing the partitions of the hard disk. It contains a lot of system utilities (parted, partimage, fstools) and basic ones (editors, midnight commander, network tools). The kernel of the system supports most important file systems (ext2/ext3, reiserfs, reiser4, xfs, jfs, vfat, ntfs, iso9660), and network ones (SMB/CIFS and NFS).
9. PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (thus, its 'Photo Recovery' name) from digital camera memory. PhotoRec ignores the filesystem and goes after the underlying data, so it will still work even if your media's filesystem has been severely damaged or re-formatted.
10. Datarescue PhotoRescue Advanced is picture and photo data recovery solution made by the creators of IDA Pro. PhotoRescue will undelete, unerase and recover pictures and files lost on corrupted, erased or damaged compact flash (CF) cards, SD Cards, Memory Sticks, SmartMedia and XD cards even when other solutions fail. Once the data is recovered, it guarantees its integrity. It supports recovery of all file types and is optimized for JPG, TIFF, GIF and BMP, as well as most camera RAW formats: CR2, RAW, RAF, CRW, NEF, ORF, MRW, etc and many types of movie files. In some cases, we can even rebuild pictures that have suffered minor corruptions.
11. MiTeC Windows Registry Recovery - crashed machine registry configuration data recovery.
    

Cryptography tools: Once the data has been collected, disks and media has been imaged, it now needs to be encrypted, hashed and digitally signed in order to be properly stored.

1. Truecrypt is a powerful open source encryption software that works on Windows (2000, 2003, XP, Vista) and Linux. It can do on the fly encryption, it can encrypt whole partitions or mass storage devices, it supports stenography (hidden volumes within an encrypted partition) for plausible deniability and supports AES-256, Serpent and Twofish encryption. It can also escrow keys (so you can't access the data without say the two USB sticks with the keys and the passphrase), and supports both password and key authentication. Note: when you're using encryption, you should also use encrypted swap or make sure you zero it out when you're done.
2. Cryptsetup-luks is an interface is based on the original cryptsetup utility and retains full compatibility, but adds extra commands to deal with the Linux Unified Key Setup (LUKS) on-disk format. This format provides additional features such as key management and key strengthening, and remembers encrypted volume configuration across reboots The Linux Unified Key Setup (LUKS) - Under Windows, LUKS encrypted disks can be used with FreeOTFE (a free, open source, "on-the-fly" transparent disk encryption program for PCs and PDAs).
3. FreeBSD GELI - cryptographic GEOM class available as of FreeBSD 6.0. The geli utility is different to gbde; it offers different features and uses a different scheme for doing cryptographic work.
4. NetBSD CGD - cryptographic device driver provides functionality which allows you to use disks or partitions for encrypted storage. After providing the appropriate key, the encrypted partition is accessible using cgd pseudo-devices.
5. OpenBSD vnconfig(8) - provides encrypted svnd's via the "vnconfig -K rounds" options associates an encryption key with the device. All data will be encrypted using the Blowfish cipher before it is written to the disk. The user is asked for both a passphrase and the name of a salt file. OpenBSD also provides encrypted swap by default.
6. OpenPGP - Open Pretty Good Privacy provides data integrity services for messages and data files by using digital signatures, encryption, compression and Radix-64 conversion. In addition, OpenPGP provides key management and certificate services. The GNU Privacy Guard (GnuPG) is the OpenPGP implementation of the GNU project. GnuPG is fully OpenPGP compliant, supports most of the optional features and provides some extra features. GnuPG is used as the standard encryption and signing tool of all GNU/Linux distributions.

Intrusion Detection and vulnerability scanners: Sometimes you need to analyze live systems during the incident. A crime has not yet been committed, or the perpetrator is still active. Intrusion detection, network monitoring and security auditing tools. Tools like AIDE, Tripwire, BART and such would need to be installed and configured before the incident occurs, but in case they are, it's good to know how to extract the right information from the hash database and figure out what files have changed and such.

Sometimes something as simple as a simple vulnerability scan using tools like nmap, Nessus and such may reveal us ways on how a perpetrator might have gained illegal access to a system.

1. AIDE - Advanced Intrusion Detection Environment is an open source intrusion detection system similar to TripWire. It maintains a database of files and hashes (md5,sha1,rmd160,tiger,haval,etc) that are used to check the integrity of a file.
2. Wireshark - Network protocol analyzer for Windows and Unix that allows examination of data from a live network, or from a capture file on disk. Also known as Ethereal (before the name change). Alternatives include tcpdump (most UNIX systems), snoop(1M) (Solaris) and Windows Network Monitor on Windows. Helps detect data theft or covert channels used to leak out information.
3. Tiger is a security tool that can be use both as a security audit and intrusion detection system. It supports multiple UNIX platforms. Unlike other tools, Tiger needs only of POSIX tools and is written entirely in shell language.
4. Lynis is an auditing tool for Unix (specialists). It scans the system configuration and creates an overview of information usable by professional auditors.
5. Rootkit Hunter scans for rootkits, backdoors and local exploits by running various tests. On Windows, you would use Sysinternals "autoruns" and "Rootkit Revealer".
6. Microsoft Baseline Security Analyzer (MBSA) is used to detect common security misconfiguration and missing security updates on your computer systems as well as to ensure the system conforms to a baseline security level.
7. BART (Basic Audit Reporting Tool) and BSM (Basic Security Module) are auditing tools used on Solaris.
8. Bastille is a hardening tool that can maintain logs and audits a forensics expert should know how to parse. Bastille is also used on HP-UX.
9. The Metasploit Framework is a development platform for creating security tools and exploits.

Antivirus and anti-spyware products: Finding malware.
The system should be scanned for trojans, keyloggers and other types of malware (make sure you only scan the image, and take no action, just log their existence).

1. Avira AntiVir PersonalEdition Classic is a free Antivirus product that is available on multiple platforms (Windows 2000 / XP / Vista 32 Bit and 64 Bit, Linux / FreeBSD / OpenBSD / Solaris). License is available for home use.
   
2. Clam AntiVirus is an open source (GPL) anti-virus toolkit for UNIX. This can be used in a commercial environment. ClamWin is a free, open source antivirus software for Microsoft Windows 98/Me/2000/XP/2003/Vista. It provides a graphical user interface to the ClamAV (Clam AntiVirus) engine.
3. AVG Anti-Virus Free Edition - only available for single computer use for home and non commercial use.
4. BitDefender 8 Free Edition is an on-demand virus scanner, which is best used in a system recovery or forensics role.
5. bdc - BitDefender Console Antivirus for FreeBSD.
6. AVG Anti-Spyware Free Edition - free antispyware solution available at no cost to home users and provides a high level of detection capability.
7. Windows Defender - Free anti-spyware tool from Microsoft.
8. Spybot Search and Destroy - Free spyware removal tool.
9. Spyware Terminator - Spyware removal tool. Can integrate ClamAV Antivirus. Free for personal and commercial use.
10. HijackThis is a free utility which quickly scans your Windows computer to find settings that may have been changed by spyware, malware or other unwanted programs. HijackThis creates a report, or log file, with the results of the scan. It generates in depth report of registry and file settings from your computer. HijackThis makes no separation between safe and unsafe settings in its scan results giving you the ability to selectively remove items from your machine. In addition to this scan and remove capability HijackThis comes with several tools useful in manually removing malware from a computer.

Malware analysis, debugging and Reverse Engineering tools:
Sometimes you happen to find rootkits, viruses, worms and all kinds of malware or software that requires the use of reverse engineering to figure out. Here's a couple of free to use tools:

1. OllyDbg is a free debugger that emphasizes binary code analysis, which is useful when source code is not available. It traces registers, recognizes procedures, API calls, switches, tables, constants and strings, as well as locates routines from object files and libraries.
2. RR0D is a ring 0 debugger. It offers the possibility to debug any kind of code (kernel/user/rasta land). Its philosophy is to be OS independent. That's why RR0D can today be installed on Linux, *BSD, Wind0ws. It's a free alternative to SoftIce.
3. Hackman Suite is a multi-module all purpose debugging tool. It includes a hex editor, a disassembler, a template editor, a hex calculator and other everyday useful tools to assist programmers and code testers with the most common tasks.
4. IDA Pro 4.9 Freeware (Interactive Disassembler) is a disassembler and debugger with lots of features that is very useful for reverse engineering. It's basically a "lite" version of the powerful IDA Pro.
5. SpyStudio is a powerful application that simplifies the code execution interception operations, also called "hooking". Users can now easily monitor and gain control over processes in their systems, to really know what is happening in the Operating System and it's applications. With SpyStudio you can monitor and intercept API calls at any time, change its parameters, and resume execution.
6. objdump is a program for displaying various information about object files. For instance, it can be used as a disassembler to view executable in assembly form. It is part of the GNU binutils for fine-grained control over executable and other binary data. It runs on multiple platforms.
7. GDB, the GNU Project debugger, allows you to see what is going on `inside' another program while it executes -- or what another program was doing at the moment it crashed.
8. GNU DDD is a graphical front-end for command-line debuggers such as GDB, DBX, WDB, Ladebug, JDB, XDB, the Perl debugger, the bash debugger, or the Python debugger. Besides ``usual'' front-end features such as viewing source texts, DDD has become famous through its interactive graphical data display, where data structures are displayed as graphs.
9. Debugging Tools for Windows are used to debug drivers, applications, and services on systems running Windows NT 4.0, Windows 2000, Windows XP, Windows Server 2003, Windows Vista, and Windows Server 2008 as well as for debugging the operating system itself. Versions of the Debugging Tools for Windows package are available for 32-bit x86, native Intel Itanium, and native x64 platforms. Debugging Tools for Windows includes WinDbg, a powerful debugger with a graphical interface and a console interface, as well as the console-based debuggers NTSD, CDB, and KD and the Windows Debugging Symbols.
10. strace (system call tracer), ltrace (library call tracer), xtrace (eXtended trace utility, similar to strace, ptrace, truss, but with extended functionality and unique features, such as dumping function calls (dynamically or statically linked), dumping call stack and more), truss (trace system calls and signals - Solaris), ktrace (enables kernel process tracing - OpenBSD) and Valgrind (executes a program under emulation, performing analysis according to one of the many plug-in modules as desired. You can write your own plug-in module as desired) are UNIX programs that let you run a program while watching the actions it performs.
    
11. DTrace is a comprehensive dynamic tracing framework for the Solaris Operating Environment (also ported to MacOS X - XRays and FreeBSD). DTrace provides a powerful infrastructure to permit administrators, developers, and service personnel to concisely answer arbitrary questions about the behavior of the operating system and user programs.

Timeline editors: analyze and evaluate the data obtained from a system and use it to determine what happened.

1. Zeitline is a Java/Swing tool that allows a computer forensic investigator to import events from various sources of a computing system or network and then order and classify them into one or more timelines of events.
2. mac-robber is a digital investigation tool that collects data from allocated files in a mounted file system. This is useful during incident response when analyzing a live system or when analyzing a dead system in a lab. The data can be used by the mactime tool in The Sleuth Kit to make a timeline of file activity. The mac-robber tool is based on the grave-robber tool from TCT and is written in C instead of Perl.

Data carving, analysis: (extracting a collection of data from a larger data set) - a file recovery technique frequently occurs during a digital investigation when the unallocated file system space is analyzed to extract files. The files are "carved" from the unallocated space using file type-specific header and footer values. File system structures are not used during the process.

1. FTimes (File Topography and Integrity Monitoring on an Enterprise Scale) is a system baselining and evidence collection tool. The primary purpose of ftimes is to gather and/or develop topographical information and attributes about specified directories and files in a manner conducive to intrusion and forensic analysis.
2. MiTeC Windows File Analyzer - tool for forensic analysis of Thumbnail Databases, Prefetch files, shortcuts, IExplore Index.DAT files and Recycle Bin contents on a Windows system. Will print a report of analyzed files.
3. Foremost is a console program to recover files based on their headers, footers, and internal data structures. Foremost can work on image files, such as those generated by dd, Safeback, Encase, etc, or directly on a drive. The headers and footers can be specified by a configuration file or you can use command line switches to specify built-in file types. These built-in types look at the data structures of a given file format allowing for a more reliable and faster recovery.
4. Scalpel is a fast file carver that reads a database of header and footer definitions and extracts matching files from a set of image files or raw device files. Scalpel is filesystem-independent and will carve files from FATx, NTFS, ext2/3, or raw partitions. It is useful for both digital forensics investigation and file recovery. Scalpel resulted from a complete rewrite of foremost 0.69, a popular open source file carver, to enhance performance and decrease memory usage.
5. ReviveIT (Revit) - smart data recovery tool (file carving).
6. Magic Rescue looks at "magic bytes" for carving out files from raw data.
   

Forensic toolkit LiveCDs:

1. Helix is Knoppix based Linux LiveCD containing various forensics tools.
2. Knoppix STD is a Linux based LiveCD containing various digital forensics tools.
   
3. The MacQuisition Boot Disk is a forensic acquisition tool used to safely and easily image Mac source drives using the source system. MacQuisition provides an intuitive user interface to the traditional command line, providing both beginner and advanced forensic examiners with a valuable tool to:
4. Winternals ERD Commander boots dead systems directly from CD into a Windows-like repair environment. It's now part of the Windows Optimization Pack.
5. BackTrack is the result of merging Auditor and Whax into a single penetration testing Linux LiveCD. It contains an assorted suite of pen-testing, data recovery and digital forensics tools.
6. SMART Linux is a Slackware based Data Forensics and Incident Response LiveCD.
7. Snarl is a FreeBSD based forensics Bootable ISO (includes Autopsy and Sleuth Kit).
8. Penguin SleuthKit Bootable CD - A Linux LiveCD that includes SleuthKit.
   
9. UBCD4Win - Ultimate Boot CD for Windows - is a bootable recovery CD based on BartPE that contains software used for repairing, restoring, or diagnosing almost computer issues. It's also easy to costomize and add various forensic tools to a Windows XP SP2 LiveCD.
10. The Farmer's Boot CD (FBCD) is a Linux boot CD designed for on-site previewing of systems before acquiring. It contains a number of programs to preview both Windows and Linux systems in a forensically sound manner. A commercial product.
    

Reference and Documentation, Whitepapers: Material you need to consult and read:

1. Bruce Schneier: His Applied Cryptography and security books as well as his blog posts and articles are a valuable resource for security experts world wide. Beyond Fear there is Bruce Schneier.
2. The Open Source Digital Forensics site is a reference for the use of open source software in digital investigations (a.k.a. digital forensics, computer forensics, incident response). Open source tools may have a legal benefit over closed source tools because they have a documented procedure and allow the investigator to verify that a tool does what it claims.
3. The Forensics Wiki, a Creative Commons-licensed wiki devoted to information about digital forensics.
4. The International Journal of Digital Evidence (IJDE)
5. NIST - National Institute of Standards and Technology - CSRC - Computer Security Division
6. DFRWS (Digital Forensics Research Workshop) is dedicated to the sharing of knowledge and ideas about digital forensics research.
7. Computer Forensics Links & Whitepapers (Forensics.nl)
8. Digital Forensics Links and Resources
9. SANS Institute - Network, Security, Computer, Audit Information Reading Room - Whitepapers on Digital Forensics
10. Forensic Focus - Computer Forensics News, Information and Community
11. The Electronic Evidence Information Center
12. Computer Forensics World - A Community of Computer Forensics Professionals
    

"Vs lbh pna ernq guvf lbh'er nyzbfg nf fzneg nf cuo."

Troff Text Processor for Typesetters

"The great strength of troff is the flexibility of the basic language and its programmability -- it can be made to do almost any formatting task."
-- Brian W. Kernighan, Rob Pike - The UNIX Programming Environment

Troff is a document processing system, and it's part of the default UNIX text processing tools.

• troff - for output device that can handle variable-width fonts and different character sizes
• nroff - produces output suitable for simple fixed-width terminals or line printers
• groff - a GNU licensed implementation of the ditroff and all the troff preprocessors
• unroff - a free flexible implementation of troff written in Scheme
  

Even though now it has been superseded by T_EX and L^AT_EX, troff is still quite common, as UNIX manual pages for the "man" command are written in troff, using the "man macros".

"Since bare troff is unusable by humans, a race of gods now gone created macro packages for mortals to use."
-- Brian Kernighan on the troff macro package.

On some systems manpages are stored in compressed formats, so, before we operate on such a manpage, we need to uncompress it.

Let's use groff and the man macros to convert a manpage to ps, and send it to the printer spooler.

% cat /usr/share/man/man7/bootparam.7.gz | gunzip | groff -man | lpr

We could also send it to ps2pdf to publish the manpage as pdf.

% cat /usr/share/man/man1/groff.1.gz | gunzip | groff -man > groff.ps && ps2pdf groff.ps && xpdf groff.pdf"

If you plan on doing anything serious with troff, you should take a look at Bell Labs Computing Science Technical Report #54, Troff User's Manual.

Problem solving with awk

"It seemed like a good idea at the time."
--Brian Kernighan

Nothing beats good old fashion awk when it comes to problem solving :-). Here is one such example:

[ircuser] is there any command or quick way to do like,say i have two strings, "tes**ng" and "lkmtide" . need to replace the '*' in the first string with the exact characters in the second string. so, "tes**ng" changes to "testing"
[ircuser] cmihai, here the issue is * is not fixed to two chars... it can be from one to len-2

# Author: Criveti Mihai, 2007.

# This program is free software.
# It comes without any warranty, to the extent permitted by applicable law.
# You can redistribute it and/or modify it under the terms of the
# Do What The Fuck You Want To Public License, Version 2
# as published by Sam Hocevar.
# See http://sam.zoy.org/wtfpl/COPYING for more details.

# Usage: cat file | awk -f change_stars_and_stuff.awk > output

BEGIN {
FS = " ";
}

{
string1 = $1;
string2 = $2;

do {
position = index( string1,"*");
finalstring = substr(string2,position,1);
sub(/\*/,finalstring,string1);
} while (position != 0)
}

{
print string1;
}

The input file would look something like this (from what I gather anyway):

cmihai@alap ~/scripts
$ cat file
tes**ng lkmtide
tes***g lkmtinaaa
testing lkmtide
abcdef* abcdefghijklmn

And now, let's take our little script for a test drive:

$ cat file | awk -f change_stars_and_stuff.awk
testing
testing
testing
abcdefg

Well, that did it :-). Good old fashion AWK to the rescue!