Making NetBSD 4.0 and OpenBSD 4.3 install in VirtualBox

Installing NetBSD 4.0 in VirtualBox fails with:
Failed to write to file. /usr/bin/dig (or whatever) (Bad address).

To make it work disable ACPI, enable VT-x and set the IDE controller type to PIIX4.

VT-x makes all the difference for NetBSD also. Without VT-x, unpacking install sets is very slow (200kb/s). With VT-x, I get 6.42 MB/s.

Plus, it won't fail writing to files.

The same thing happens with OpenBSD, and can be fixed by using VT-x.

OpenBSD Web Portal server (Drupal)

Configuring an OpenBSD Web Portal server Running Drupal on chrooted Apache + MOD_SSL + PHP + MySQL or PostgreSQL

Final goal: Drupal Web portal with TinyMCE JavaScript editor, Unicode and Locales support running on OpenBSD 4.3.

1. Installing and configuring OpenBSD – read the FAQ, partition.

a. You can install OpenBSD via cdrom (install43.iso or cd43.iso for netinstall) or netboot via TFTP / BOOTP / DHCPD. Read diskless(8) for details. On Windows, you can use TFTPD32 for DHCP/TFTP.

b. Partition the disk (read the FAQ on partitioning). Example partitioning scheme:

Partition	Mount location	Filesystem	Mount options
/dev/sd0a	/	ffs	rw,softdep	1 1
/dev/sd0b	swap	swap
/dev/sd0c	Whole disk
/dev/sd0d	/tmp	ffs	rw,softdep,nodev,nosuid	1 2
/dev/sd0e	/var	ffs	rw,softdep,nodev,nosuid	1 2
/dev/sd0f	/usr	ffs	rw,softdep,nodev	1 2
/dev/sd0g	/home	ffs	rw,softdep,nodev,nosuid	1 2

c. Disable non-critical services (like ident, time, daytime, etc.) from /etc/inetd.conf. Just comment out the lines you don’t want with a #.

d. Make sure you enable Soft Updates (softdep) mounts – they increase the Filesystem performance tenfold when it comes to writing a lot of small files. This is done by adding “softdep” to mount options in /etc/fstab.

e. Add a regular user account. You can use the useradd(8) script. Make sure you add the user to the wheel group to allow su – root.

f. For added security configure „sudo” by running visudo(8). You can permit all users in the wheel group to sudo root: %wheel ALL=(ALL) NOPASSWD: SETENV: ALL

i. It’s usually best to disable root logins and use “sudo” for root access (sudo –i or su – if you need a root sheel). Edit /etc/ssh/sshd_config. You can also change the default ssh port from 22 to some random port to mitigate against brute force attacks. Keeps them from filling out your logs with junk. Consider using ssh keys also.

ii. Port 6969

iii. PermitRootLogin no

iv. Restart the OpenSSH daemon: „pkill –HUP sshd”

g. Configure your user profile to use a ftp mirror. Add the following to your ~/.profile: (or whatever shell you’re using, eg: .bashrc, .zshrc): PKG_PATH=ftp://ftp.su.se/pub/OpenBSD/4.3/packages/i386/; export PKG_PATH

h. Restart the system (to mount softdep, etc.).

i. Install some “critical” packages:

i. pkg_add -vi mc zsh screen elinks zip unzip bzip2

j. Install ports

i. cd /usr
sudo ftp ftp://ftp.su.se/pub/OpenBSD/4.3/ports.tar.gz
sudo tar zxf ports.tar.gz

k. Unpack the OpenBSD source code (src and kernel – sys) – these are required for patching OpenBSD:

i. cd /usr/src
sudo ftp ftp://ftp.su.se/pub/OpenBSD/4.3/src.tar.gz
sudo ftp ftp://ftp.su.se/pub/OpenBSD/4.3/sys.tar.gz
sudo tar zxvf src.tar.gz
sudo tar zxvf sys.tar.gz

l. Apply OpenBSD security patches. You download patches from http://www.openbsd.org/errata43.html and you apply them using the instructions in the patch headers:

i. ftp ftp://ftp.openbsd.org/pub/OpenBSD/patches/4.3.tar.gz
You can put 001_openssh.patch, 002_openssh2.patch and the following patches in /usr/src and apply them. Read the header for the patches for instructions.

% head -11 001_openssh.patch
Apply by doing:
cd /usr/src
patch -p0 < install ="="> lib
===> ssh
install -c -s -o root -g bin -m 555 ssh /usr/bin/ssh
install -c -o root -g bin -m 444 ssh.cat1 /usr/share/man/cat1/ssh.0
install -c -o root -g bin -m 444 ssh_config.cat5 /usr/share/man/cat5/ssh_config.0
/usr/share/man/cat1/slogin.0 -> /usr/share/man/cat1/ssh.0
/usr/bin/slogin -> /usr/bin/ssh
===> sshd
install -c -s -o root -g bin -m 555 sshd /usr/sbin/sshd
install -c -o root -g bin -m 444 sshd.cat8 /usr/share/man/cat8/sshd.0
install -c -o root -g bin -m 444 sshd_config.cat5 /usr/share/man/cat5/sshd_config.0
===> ssh-add
install -c -s -o root -g bin -m 555 ssh-add /usr/bin/ssh-add
install -c -o root -g bin -m 444 ssh-add.cat1 /usr/share/man/cat1/ssh-add.0
===> ssh-keygen
install -c -s -o root -g bin -m 555 ssh-keygen /usr/bin/ssh-keygen
install -c -o root -g bin -m 444 ssh-keygen.cat1 /usr/share/man/cat1/ssh-keygen.0
===> ssh-agent
install -c -s -o root -g _sshagnt -m 2555 ssh-agent /usr/bin/ssh-agent
install -c -o root -g bin -m 444 ssh-agent.cat1 /usr/share/man/cat1/ssh-agent.0
===> scp
install -c -s -o root -g bin -m 555 scp /usr/bin/scp
install -c -o root -g bin -m 444 scp.cat1 /usr/share/man/cat1/scp.0
===> sftp-server
install -c -s -o root -g bin -m 555 sftp-server /usr/libexec/sftp-server
install -c -o root -g bin -m 444 sftp-server.cat8 /usr/share/man/cat8/sftp-server.0
===> ssh-keysign
install -c -s -o root -g bin -m 4555 ssh-keysign /usr/libexec/ssh-keysign
install -c -o root -g bin -m 444 ssh-keysign.cat8 /usr/share/man/cat8/ssh-keysign.0
===> ssh-keyscan
install -c -s -o root -g bin -m 555 ssh-keyscan /usr/bin/ssh-keyscan
install -c -o root -g bin -m 444 ssh-keyscan.cat1 /usr/share/man/cat1/ssh-keyscan.0
===> sftp
install -c -s -o root -g bin -m 555 sftp /usr/bin/sftp
install -c -o root -g bin -m 444 sftp.cat1 /usr/share/man/cat1/sftp.0
===> scard
install -c -m 444 -o root -g bin Ssh.bin /usr/libdata/ssh

• 2. Restart the system to apply patches (or just the affected services by using pkill –HUP for example if you didn’t patch the kernel).

3. Installing additional OpenBSD filesets if required by say php5-gd: (if you skipped this at install). You can read the FAQ, but this is basically it:

a. % cd / && sudo ftp ftp://ftp.su.se/pub/OpenBSD/4.3/i386/xbase43.tgz
% sudo tar xzvphf xbase43.tgz

% sudo ldconfig -m /usr/X11R6/lib

4. Install PHP 5 (and various libraries for image manipulation, database access and Unicode support):

a. % sudo pkg_add -vi php5-core php5-mysql php5-curl php5-mbstring php5-gd
Ambiguous: choose package for php5-gd
0:
1: php5-gd-5.2.5
2: php5-gd-5.2.5-no_x11
Your choice: 1
parsing php5-gd-5.2.5

b. Activate PHP modules by creating the various symbolic links:

i. % sudo ln -s /var/www/conf/modules.sample/php5.conf /var/www/conf/modules

ii. % sudo ln -fs /var/www/conf/php5.sample/curl.ini /var/www/conf/php5/curl.ini

iii. % sudo ln -fs /var/www/conf/php5.sample/gd.ini /var/www/conf/php5/gd.ini

iv. % sudo ln -fs /var/www/conf/php5.sample/mbstring.ini /var/www/conf/php5/mbstring.ini

v. % sudo ln -fs /var/www/conf/php5.sample/mysql.ini /var/www/conf/php5/mysql.ini

5. Install, configure and secure the MySQL database:

a. % sudo pkg_add -vi mysql-server

b. Installing the default database

i. % sudo /usr/local/bin/mysql_install_db

c. Starting the MySQL service:

i. % sudo /usr/local/bin/mysqld_safe&
[1] 32390
% Starting mysqld daemon with databases from /var/mysql

d. Secure the installation (delete anonymous users, the test database and set a root password). PS: don’t use # in your password, there’s a bug in the script. Set that manually if you want.

i. % sudo /usr/local/bin/mysql_secure_installation

e. Tune some sysctl parameters for MySQL:

i. Edit /etc/sysctl.conf:
kern.shminfo.shmall=32768
kern.maxfiles=8192

ii. Apply the changes
% sudo sysctl kern.shminfo.shmall=32768
kern.shminfo.shmall: 8192 -> 32768
% sudo sysctl kern.maxfiles=8192
kern.maxfiles: 3580 -> 8192

iii. Add a mysql login in /etc/login.conf:
mysql:\
:openfiles-cur=1536:\
:openfiles-max=3096:\
:tc=daemon:

iv. % sudo cap_mkdb /etc/login.conf

6. Configure SSL (Secure Sockets Layer) for Apache mod_ssl (https). Generate a self signed certificate (or sign one) by reading ssl(8).

a. % sudo openssl genrsa -out /etc/ssl/private/server.key 1024
% sudo openssl req -new -key /etc/ssl/private/server.key -out /etc/ssl/private/server.csr
% sudo openssl x509 -req -days 365 -in /etc/ssl/private/server.csr -signkey /etc/ssl/private/server.key -out /etc/ssl/server.crt

b. Test SSL support (lynx is ssl enabled):
% sudo apachectl startssl
% lynx https://localhost

7. Configure the Apache webserver:

a. Edit the Apache configuration file and setup PHP and server details /var/www/conf/httpd.conf
AddType application/x-httpd-php .php
ServerAdmin cmihai@website
DirectoryIndex index.html index.php
ServerName hostname(fqdn)

b. Configure PHP:

i. Edit the PHP config /var/www/conf/php.ini
upload_max_filesize = 12M

c. Test PHP:
ed /var/www/htdocs/index.php
a

Test PHP:
.
w
q
% lynx http://localhost/index.php

d. Configure Sendmail for supporting the Apache chroot[1]
% sudo pkg_add -vi mini_sendmail-chroot
parsing mini_sendmail-chroot-1.3.6p0
mini_sendmail-chroot-1.3.6p0: complete

% sudo mkdir -p /var/www/usr/sbin/
% sudo ln /var/www/bin/mini_sendmail /var/www/usr/sbin/sendmail
% sudo cp /bin/sh /var/www/bin

8. Configure services to start at boot (Apache and MySQL):

a. % sudo ed /etc/rc.conf
httpd_flags="-DSSL"

b. % sudo ed /etc/rc.local
if [ -x /usr/local/bin/mysqld_safe ] ; then
su -c mysql root -c '/usr/local/bin/mysqld_safe >/dev/null 2>&1 &'
echo -n ' mysql'
fi
# Crearea unui link pentru socketul MySQL in chrootul Apache
mkdir -p /var/www/var/run/mysql
sleep 5
ln -f /var/run/mysql/mysql.sock /var/www/var/run/mysql/mysql.sock

c. Reboot to test changes.

9. Crate a database for Drupal and a user. Grand permissions:

a. % sudo mysql -u root -p
CREATE USER drupal IDENTIFIED BY 'password';
CREATE DATABASE drupal CHARACTER SET utf8;
GRANT ALL PRIVILEGES ON drupal.* to USER drupal@localhost IDENTIFIED BY 'password';

10.Install Drupal and modules:

a. % sudo pkg_add -vi drupal5-tinymce drupal5-link-to-content drupal5-imce drupal5-image drupal5-backup-migrate drupal5-autolocale drupal5-token drupal5-securelogin ImageMagick

11.Optionally configure .httauth for security:

a. % cd /var/www
% sudo htpasswd -c htpasswd username

Add .htaccess to /var/www/htdocs
AuthName "Nu este permis accesul"
AuthType Basic
AuthUserFile ../htpasswd
require valid-user

b. Add “Add AllowOverride All” în /var/www/conf/httpd.conf.

c. Restart Apache (apachectl stop && apachectl startssl).

You can now configure Drupal and the various modules. Be sure to setup a backup schedule. You can use the Drupal database backup and restore module.

You can also install Apache mod_security for additional security:
% sudo pkg_add -vi modsecurity-apache
% sudo /usr/local/sbin/mod_security-enable
% sudo apachectl stop && sudo apachectl startssl

Make sure you read /usr/local/share/doc/mod_security/modsecurity-manual.pdf

You should also keep an eye on your webserver. For monitoring, I recommend top(1), vmstat, (8), sysctl hw, pftop and mytop.

% sudo pkg_add -vi mytop pftop
% sudo /usr/local/sbin/pftop

To use mytop you'll need a ~/.mytop file (and set proper permissions on it). You can add something like:

prompt=1
pass=
user=drupal
db=drupal
delay=5
port=3306
socket=
batchmode=0
header=1
color=1
idle=1

% mytop


MyTOP in Action

You'll also want to setup Packet Filter. Read the PF User's Guide, pf.conf(5) and the example rulesets in /usr/share/pf/examples. Enable pf in /etc/rc.conf (pf=YES).

---

[1] Read chroot(2) ssl(8) afterboot(8), etc.

OpenBSD 4.3 has been released

OpenBSD 4.3 has been released.

• Changelist
  
• Mirror list.
• Patches
• Upgrade Guide
• FAQ
  

DragonFlyBSD - HAMMER Filesystem

The DragonFlyBSD HAMMER Filesystem has made progress. DragonFlyBSD 2.0 is expected shortly, so it may well be worth a look.

OpenBSD "distribution" for embedded hardware

Flashboot is a system built by Damien Miller and others as an adaptation of OpenBSD that's more suited for small flash-based hardware (like the like Soekris or Wrap machines). For most applications you don't have to compile it on you own, you just put the binary release on a flashcard and you're set (somewhat simplified).

OpenBSD automated / unattended installation - Yaifo

Don't have remote servers with KVM? Well, this will help:

Yaifo has been updated for OpenBSD 4.2 and is now actively maintained by Mike Erdely. You can download it here:
http://erdelynet.com/category/tech/yaifo/

It's basically a modified installer with ssh support, RAID support and all that. You can basically do remote, scripted, unattended OpenBSD installations, etc.

NetBSD 4.0 Released, 54 platforms supported.

NetBSD 4.0 Release announcement.

Major changes in NetBSD 4.0:

• Xen 3
• Blueetooth Support
• iSCSI support
• CARP - The (OpenBSD) Common Address Redundancy Protocol
• mprotect(2) to enforce OpenBSD's W^X policies
• Kernel Authorization Framework
• Veriexec file integrity subsystem

Download NetBSD from one of the mirror sites:
http://www.netbsd.org/mirrors/

Also, consider donating to the NetBSD project:
http://www.netbsd.org/donations/

OpenBSD and USB audio cards

If you have a USB audio card - uaudio(4) you must configure OpenBSD to use the proper sound device:

$ tail -f /var/log/messages

Plug in your USB audio card...

Use usbdevs(8) and dmesg(8) to figure out what audio devices it uses...

Configure /dev/audio, /dev/audioctl, /dev/mixer to point to /dev/audio1 or whatever dmesg reports as your new audio card and check if everything is OK;

$ ls -l /dev/audio* /dev/audioctl* /dev/mixer* /dev/sound* && audioctl -a && mixerctl -a"

Test your sound:

"cat /dev/urandom > /dev/audio"

Enjoy.

QNX Releases Source Code for New Networking Stack based on NetBSD

New QNX Neutrino Real Time Operating System Network Stack source code available, based on NetBSD code.

"Based largely upon the NetBSD4 code base, the stack includes a number of new features that address security, enhance speed, and aid in code portability."

Article on CNN Money

UNIX Deployment Tools - JumpStart, IgniteUX, NIM, KickStart, AutoYaST, FAI

Bare metal recovery and mass deployment tools for UNIX and UNIX-like systems:

On Windows there's RIS, WDS or tools like Ghost, on UNIX platforms we have tools like JumpStart, IgniteUX, NIM, FAI, KickStart, etc. to help with massive deployment of operating systems.

UNIX:

• Sun Solaris - Custom JumpStart and Advanced Installations - The custom JumpStart installation method is a command–line interface that enables you to automatically install or upgrade several systems, based on profiles that you create. The profiles define specific software installation requirements. You can also incorporate shell scripts to include preinstallation and postinstallation tasks. You choose which profile and scripts to use for installation or upgrade. The custom JumpStart installation method installs or upgrades the system, based on the profile and scripts that you select. Also, you can use a sysidcfg file to specify configuration information so that the custom JumpStart installation is completely hands-off.
• Sun Solaris - JumpStart Enterprise Toolkit: provides a framework to simplify and extend the JumpStart functionality provided within the Solaris Operating System.
  
• Sun Solaris Flash Archives (flar) - can be used with JumpStart technology to automate and speed up deployment or disaster recovery.
  
• HP HP-UX Ignite-UX - is an administration toolset that allows: Simultaneous installation of HP-UX on multiple clients, The creation and use of custom installations, The remote recovery of clients, The creation of recovery media.
• IBM AIX mksysb/mkcd/mkdvd: The mksysb command creates a backup of the operating system (that is, the root volume group). You can use this backup to reinstall a system to its original state after it has been corrupted. If you create the backup on tape, the tape is bootable and includes the installation programs needed to install from the backup.
• IBM AIX NIM - Network Installation Management - is an excellent feature of the AIX operating system and is very important for teams or companies that have a need to install or upgrade many RS/6000 machines with the same images at the same time. NIM supports the use of mksysb images. Performing a NIM mksysb installation is faster than performing a NIM rte installation, and with mksysb, you can optionally include other installed software. You can use a mksysb image to install the nodes of a CSM cluster.

Linux:

• RedHat Linux Kickstart provides automation of Linux installation that uses a single kickstart file to install the system on multiple machines.
• SUSE Linux AutoYaST - Automatic Linux Installation and Configuration with YaST2. AutoYaST allows unattended and automated installation. With AutoYaST, administrators can create a consistent baseline configuration for new installations in large or expanding deployments. In addition to AutoYaST, other installation methods include PXE Boot, CD-ROM, NFS, CIFS/SMB, HTTP, FTP, and the Service Location Protocol (SLP), which allows autodetection of install servers. ALICE, SuSEs former auto-installation system was a system built around the auto-installation features that were available with YaST1. In order to be able to use existing ALICE configuration files and resources, a special option is provided in the configuration system will let you convert ALICE configuration files into a control file readable by AutoYaST.
• Debian GNU/Linux FAI - Fully Automatic Installation - is an automated installation tool to install or deploy Debian GNU/Linux and other distributions on a bunch of different hosts or a Cluster. FAI can also be used for configuration management of a running system.
  

BSD:

• Automatic OpenBSD Installation - Jumpstart-style procedure for installing OpenBSD servers
• FreeBSD "JumpStart" Guide - This article details the method used to allow machines to install FreeBSD using the Intel PXE method of booting a machine over a network. Use sysinstall install.cfg for scripting.
• BSD PXEBoot - while not unassisted, BSD systems can easily boot from PXE and install over the network.
  

Tools:

• Cfengine - an adaptive system configuration management engine - is an automated suite of programs for configuring and maintaining Unix-like computers. It has been used on computing arrays of between 1 and 20000 nodes.
  

qemu - kqemu accelerator ported to other platforms

QEMU is a generic and open source machine emulator and virtualizer. When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. A host driver called the QEMU accelerator (also known as KQEMU) is needed in this case. The virtualizer mode requires that both the host and guest machine use x86 compatible processors.

QEMU Accelerator (KQEMU) is a driver allowing the QEMU PC emulator to run much faster when emulating a PC on an x86 host. It basically makes qemu speeds comparable to those of VirtualBox or VMware.

KQemu has been ported to various platforms (x86):

• FreeBSD kqemu
• OpenBSD kqemu
• Solaris kqemu
• Windows kqemu also see Qemu Windows
• Linux kqem

DTrace: ported to FreeBSD, MacOS X, QNX

DTrace is being ported to QNX Neutrino! After seeing DTrace ported to FreeBSD, MacOS X Leopard, and hearing hints about NetBSD ports, I still find the QNX port surprising.

Master bootable OpenBSD 4.2 install media

Although referenced in the documentation:

cdrom42.fs The i386 boot and installation 2.88MB
floppy image that contains almost all OpenBSD
drivers; see below.

the cdrom42.fs image is no longer included in OpenBSD. If you want to make your own custom OpenBSD cd without having to manipulate the 200MB iso, you can simply use the older cdrom41.fs:

See http://unixsadm.blogspot.com/2007/08/master-openbsd-iso-image.html for details, but use the 4.2 install sets.

mkisofs -vrTJ -V "OpenBSD42" -b 4.1/i386/cdrom41.fs -c boot.catalog -o OpenBSD.iso ~/OpenBSD

Yep, you can use cdrom41.fs and it will just work.

OpenBSD 4.2 Release, ISO image with install sets available

The OpenBSD project has released OpenBSD 4.2 adding new and improved platform support (hppa, sparc64, alpha), performance enhancements and new security features and is available for download on the FTP mirrors. A detailed log of changes is available here.

The OpenBSD 4.2 release song, entitled "100001 1010101" has also been available for some time now.

As of OpenBSD 4.2 you can now download a ISO image containing the install sets. The install42.iso image (md5 03dc43a1d18d3003843a1f13b3861917) can now be downloaded and simply burned to a CD. You can get the image here:
ftp://ftp.su.se/pub/OpenBSD/4.2/i386/install42.iso or on any other FTP mirror.

Digital Forensic Tools: Imaging, Virtualization, Cryptanalysis, Steganalysis, Data Recovery, Data Carving, Reverse Engineering

"Jrypbzr gb gur bgure fvqr."

Computer Forensics is a science and an art. And to perform it, you need tools to identify, acquisition, preserve and analyze data in a clean, safe, non-destructive manner. Lots of tools. Everything from data acquisition to virtualization and steganalysis.

A list of more or less free tools (mostly open source or freeware, but I have included some relevant commercial products) no digital forensics expert should be without:

Data acquisition, enumeration, imaging and forensics tools: Toolkits and utilities.

1. The Sleuth Kit and Autopsy Browser. Both are open source digital investigation tools (a.k.a digital forensic tools) that run on Unix systems (such as Linux, OS X, FreeBSD, OpenBSD, and Solaris). They can be used to analyze NTFS, FAT, Ext2, Ext3, UFS1, and UFS2 file systems and several volume system types. The Sleuth Kit (TSK) is a collection of command line tools based on The Coroner's Toolkit (TCT). Autopsy is a graphical interface to TSK.
2. The Coroner's Toolkit (TCT) is a forensics toolkit for analysis of UNIX break-ins. It runs on BSD (OpenBSD, FreeBSD, BSD/OS), Solaris/SunOS, Linux and HP-UX.
   
3. WinHex: Computer Forensics & Data Recovery Software, Hex Editor & Disk Editor.
4. dcfldd is an enhanced version of GNU dd with features useful for forensics and security. GNU ddrescue is a data recovery tool. It copies data from one file or block device trying to rescue data in case of read errors. It's a better alternative to using dd_rescue and dd_rhelp or SpinRite (you can just do a disk refresh with dd: "dd if=/dev/disk of=/dev/disk bs=2m" while the drive isn't mounted - no write operations going on - or something along those lines in order to prevent presently recoverable read errors from progressing into unrecoverable read errors).
5. Sysinternals tools contains programs like streams that help us find data hidden inside alternate streams or strings that grep readable strings from a file. It also has tools like process explorer, procmon, autoruns and rootkit revealer that allow you to dig deep into the Windows operating system to process, disk and data related information.
   
6. Microsoft Log Parser is a powerful, versatile tool that you can use to extract information from files of almost any format by using Structured Query Language to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory.
7. AccessData Forensic Products: FTK - Forensic Toolkit, Registry Viewier - more neat tools from AccessData. Commercial products.
8. Clonezilla is used to clone many computers simultaneously. It can perform a full disk image or just file backup. It's a backup tool, but it can also perform bit by bit disk imaging.
9. Sysinternals LiveKD is a Live version of Windows Debugger (WinDBG) that allows you to run the Kd and Windbg Microsoft kernel debuggers, which are part of the Debugging Tools for Windows package, locally on a live system. This means that you to easily take a memory dump of a running Windows system (.dump /f YOURUSBDISK:\fullmemorydump.dmp). On UNIX systems you can use dd to take a snapshot of the system memory ("dd if=/dev/kmem of=/path/to/memorydump").
10. Paraben's Device Seizure - Cell Phone and PDA Forensic software. Specialized software for portable device forensics.
11. PDD is a forensic analysis tool for Palm OS platform devices. It is an open source Windows-based tool for Palm OS memory imaging and forensic acquisition. The Palm OS Console Mode is used to acquire memory card information and to create a bit-for-bit image of the selected memory region. No data is modified on the target device and the data retrieval is not detectable by the user of the PDA.
12. CDInfo is an application that will display all ISO descriptors from all attached cd-rom drives (Label, System, Application, VolumeSet, Copyright, Creation Date, Directory Start, Directory Length, extentions, tracks, etc).
13. PMDump is a Windows tool that lets you dump the memory contents (both RAM and swap) of a process to a file without stopping the process.

Virtualization: Once the actual machine is cloned, it's usually put inside a virtual machine (features like snapshots and debugging help quite a bit with the digital forensics process). This is called physical to virtual (P2V) migration.

1. Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to "boot up" the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra "throw away" copies of the disk or image to create the virtual machine.
2. VMware Converter Starter is a free p2v (physical to virtual) migration tool. VMware Converter quickly converts Microsoft Windows based physical machines and third party image formats to VMware virtual machines. It also converts virtual machines between VMware platforms. Note: for digital forensic images, you should use LiveView, and not the converter.
   
3. VMware Server allows for free virtualization. You can use it in combination with Live View to virtualize existing environments, and use the snapshots feature to revert back to a previous state of a virtual machine in an instant.
4. QEmu - a much more flexible virtualization program, albeit a bit slower than VMware. Supports emulating IA-32 (x86) PCs, AMD64 PCs, MIPS R4000, Sun's SPARC sun4m, Sun's SPARC sun4u, ARM development boards (Integrator/CP and Versatile/PB), SH4 SHIX board, PowerPC (PReP and Power Macintosh), and ETRAX CRIS architectures. Also, qemu-img can be a valuable tool for converting virtual machine images. Also allows for some really low level debugging features. A modified version of QEmu can even emulate PIX platforms (or Juniper JunOS systems like Olive).
   
5. VirtualBox is a GPL licensed x86 virtualization platform that runs on Windows, Linux and MacOS hosts, and supports various x86 client machines (Windows, Linux, BSD, Solaris). It's a noteworthy alternative to using VMware, as performance tends to be pretty good.
6. Microsoft VirtualServer / Virtual PC are free virtualization products from Microsoft. They support all major features (snapshots, mounting ISO images and such), and performance tends to be reasonable (to some extent, similar to that of VMware).
7. SIMH is a highly portable, multi-system simulator. It can emulate VAX and PDP-11 platforms. Just in case you need to perform forensics on older minicomputers.
8. Hercules is an open source (QPL licensed) emulator of IBM Mainframe computers (System/370, ESA/390 architectures and even the 64-bit zSeries). Hercules runs under Linux, Windows (98, NT, 2000, and XP), FreeBSD, and Mac OS X (10.3 and later). Hercules will run OS/360, DOS/360, DOS/VS, MVS, VM/370, TSS/370 - all IBM public domain operating system, as well as OS/390, z/OS, VSE/ESA, z/VSE, VM/ESA, and z/VM, and even Linux/390 and Linux (SuSE, RHEL, Debian, CentOS and Slackware) on zSeries.
9. Oracle VM is a server virtualization software based on Xen and Oracle Linux (itself based on RHEL sources) that fully supports both Oracle and non-Oracle applications. It is a free alternative to VMware Virtual Infrastructure (VMware ESX + VirtualCenter). It is certified to run the Linux operating system, Oracle Database, Fusion Middleware, and Application software, thus makes a very good platform for investigating Oracle databases.
10. The Palm OS Emulator is a program based on the Copilot app that emulates the hardware of the various models of Palm-powered handhelds, making it a valuable tool for writing, testing, and debugging applications as well as obtainiwinng evidence from the device.
11. Microsoft Windows CE 5.0 Device Emulator contains the emulator technologies featured in Windows CE 5.0. By using the Device Emulator, you can run emulated-based images created by Windows CE 5.0 without installing Platform Builder, its platform development tool.
    

Password recovery tools: You may often need to recover keys and passwords.

"This text has been encrypted twice... for double protection!"

1. Ophcrack is a very efficient Windows password cracker based on rainbow tables. It will crack huge tables of LM hashes in under 3 minutes. It also comes in the form of a LiveCD (though in digital forensics cases it's usually best to extract the SAM file containing the password hashes from the disk image and use that. Ophcrack can be a lot more effective if you have more complete rainbow tables.
2. LCP is a free alternative to the now dead L0phtcrack.
3. John the Ripper is a very versatile password cracking tool. It's supported on different architectures and operating systems (UNIX, Windows, OpenVMS, etc) and it's quite fast. Besides several crypt(3) password hash types most commonly found on various Unix flavors, supported out of the box are Kerberos AFS and Windows NT/2000/XP/2003 LM hashes, plus several more with contributed patches.
4. Medusa is a very fast parallel brute force login password cracker.
5. Elcomsoft Password Recovery suite: anything from office, archives, pdf files etc. to email clients. These are commercial products though.
6. CmosPwd decrypts password stored in CMOS used to access BIOS SETUP. Works on a lot of BIOSes (AMI, Award, Phoenix, IBM, etc). It can also be used to backup, restore or erase the BIOS.
7. AccessData Decryption Tools: PRTK - Password Recovery Toolkit, DNA - Distributed Network Attack, PORT - Portable Office Rainbow Tables are some of the best and fastest tools in the business.
8. Offline NT Password and Registry editor - a utility to (re)set the password of any user that has a valid (local) account on your NT system. You do not need to know the old password to set a new one. Features a registry editor. Supports 32 and 64 versions of Vista (and NTFSv5).
9. Elcomsoft Distributed Password Recovery is designed for distributed recovery of forgotten or lost passwords of different documents. Version 2.0 adds support for Windows SYSKEY startup passwords, passwords stored in Domain Cached Credentials, includes updated Adobe Acrobat module, and provides hardware acceleration (now up to 25 times faster!) for NTLM password recovery using GeForce 8 video cards.
10. Dialupass - Dialup Password Recovery - Recovers the passwords of dialup entries (VPN and Internet connections) on Windows systems. NirSoft provides a couple of free password recovery tools for various products such as Instant Messaging applications, cached passwords stored by Internet Browsers, E-mail clients and so on.

Here's a little cool trick for recovering cached passwords (asterisk passwords) stored in your Internet Browser (Firefox, Opera, Internet Explorer or anything with JavaScript).

Steganalysis and stenography: how to detect hidden data using stenography.

1. Stegdetect finds hidden information in JPEG images using such steganography schemes as F5, Invisible Secrets, JPHide, and JSteg. XSteg is a GUI.
2. Stego Suite is a powerful commercial Stenography detection toolkit, consisting of 3 major tools.
3. Stegkit is an Automated Steganalysis Tool.
4. Digital Invisible Ink Toolkit is an open-source cross-platform image steganography suite that includes both steganography and steganalysis implementations.
5. StegSpy will detect steganography and the program used to hide the message.
   
6. SteGUI is a StegHide GUI.
7. Digital Watermarking allows you to hide copywrite information and such in media (images and such) that's present even after encoding to another format (bmp->jpg), printing, copy/paste, etc. You can use ImageMagick of various Photoshop plugins to do this.
8. Stepic is a Python module and command line tool for hiding arbitrary data within images by slightly modifying the colors. These modifications are generally imperceptible to humans, but are machine detectable.
9. wbStego4 offers steganography in bitmaps, text files, HTML files and PDF files. It is has two very user-friendly interfaces and is ideal for securely transmitting data online or adding copyright information, especially with the copyright information manager.
   
10. NL Stego is a system for text generation and text-based steganography. It combines Markov Models of several orders to generate random text resembling a given training text (or text corpus). It can also embed secret messages into pseudo-random generated text.
11. Steghide is an Open Source (GPL) steganography program that is able to hide data in various kinds of image- and audio-files. The color- respectivly sample-frequencies are not changed thus making the embedding resistant against first-order statistical tests. Supports compression of embedded data, encryption of embedded data, embedding of a checksum to verify the integrity of the extraced data and has support for JPEG, BMP, WAV and AU files.
    
12. StegFS is an Open Source (GPL) Steganographic File System for Linux. Not only does it encrypt data, it also hides it such that it cannot be proved to be there.
    

Filesystem tools: Data Recovery.

1. Undelete Plus is a free deleted file recovery tool that works for all versions of Windows (95-Vista), FAT12/16/32, NTFS and NTFS5 filesystems and can perform recovery on various solid state devices.
2. Avira UnErase Personal - a freely available unerase product.
   
3. TestDisk is a free (GPL) data recovery software that can fix partition tables, recover deleted partitions and rebuild NTFS boot sectors. It can find lost partitions (anything from BSD disklabels to IBM JFS, it supports pretty much anything).
   
4. GNU Parted is a program for creating, destroying, resizing, checking, and copying partitions, and the file systems on them. This is useful for creating space for new operating systems, reorganising hard disk usage, copying data between hard disks, and disk imaging. It can also be used to attempt recovery of the partition table similar to TestDisk (rescue START END).
   
5. Stellar Phoenix has various UNIX and *NIX (SCO OpenServer, Unixware, Sun Solaris, *BSD, HP-UX, MacOS) data recovery tools as well as some Windows Data Recovery tools. They are, however, commercial products.
6. R-Studio is a data recovery software suite that can recover files from FAT(12-32), NTFS, NTFS 5, HFS/HFS+, FFS, UFS/UFS2 (*BSD, Solaris), Ext2/Ext3 (Linux) and so on. This is a commercial product.
7. DiskInternals NTFS Recovery is a fully automatic utility that recovers data from damaged or formatted disks. This is a commercial product.
8. SystemRescueCD is a Linux system on a bootable CD/DVD for repairing your system and your data after a crash. It also aims to provide an easy way to carry out admin tasks on your computer, such as creating and editing the partitions of the hard disk. It contains a lot of system utilities (parted, partimage, fstools) and basic ones (editors, midnight commander, network tools). The kernel of the system supports most important file systems (ext2/ext3, reiserfs, reiser4, xfs, jfs, vfat, ntfs, iso9660), and network ones (SMB/CIFS and NFS).
9. PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (thus, its 'Photo Recovery' name) from digital camera memory. PhotoRec ignores the filesystem and goes after the underlying data, so it will still work even if your media's filesystem has been severely damaged or re-formatted.
10. Datarescue PhotoRescue Advanced is picture and photo data recovery solution made by the creators of IDA Pro. PhotoRescue will undelete, unerase and recover pictures and files lost on corrupted, erased or damaged compact flash (CF) cards, SD Cards, Memory Sticks, SmartMedia and XD cards even when other solutions fail. Once the data is recovered, it guarantees its integrity. It supports recovery of all file types and is optimized for JPG, TIFF, GIF and BMP, as well as most camera RAW formats: CR2, RAW, RAF, CRW, NEF, ORF, MRW, etc and many types of movie files. In some cases, we can even rebuild pictures that have suffered minor corruptions.
11. MiTeC Windows Registry Recovery - crashed machine registry configuration data recovery.
    

Cryptography tools: Once the data has been collected, disks and media has been imaged, it now needs to be encrypted, hashed and digitally signed in order to be properly stored.

1. Truecrypt is a powerful open source encryption software that works on Windows (2000, 2003, XP, Vista) and Linux. It can do on the fly encryption, it can encrypt whole partitions or mass storage devices, it supports stenography (hidden volumes within an encrypted partition) for plausible deniability and supports AES-256, Serpent and Twofish encryption. It can also escrow keys (so you can't access the data without say the two USB sticks with the keys and the passphrase), and supports both password and key authentication. Note: when you're using encryption, you should also use encrypted swap or make sure you zero it out when you're done.
2. Cryptsetup-luks is an interface is based on the original cryptsetup utility and retains full compatibility, but adds extra commands to deal with the Linux Unified Key Setup (LUKS) on-disk format. This format provides additional features such as key management and key strengthening, and remembers encrypted volume configuration across reboots The Linux Unified Key Setup (LUKS) - Under Windows, LUKS encrypted disks can be used with FreeOTFE (a free, open source, "on-the-fly" transparent disk encryption program for PCs and PDAs).
3. FreeBSD GELI - cryptographic GEOM class available as of FreeBSD 6.0. The geli utility is different to gbde; it offers different features and uses a different scheme for doing cryptographic work.
4. NetBSD CGD - cryptographic device driver provides functionality which allows you to use disks or partitions for encrypted storage. After providing the appropriate key, the encrypted partition is accessible using cgd pseudo-devices.
5. OpenBSD vnconfig(8) - provides encrypted svnd's via the "vnconfig -K rounds" options associates an encryption key with the device. All data will be encrypted using the Blowfish cipher before it is written to the disk. The user is asked for both a passphrase and the name of a salt file. OpenBSD also provides encrypted swap by default.
6. OpenPGP - Open Pretty Good Privacy provides data integrity services for messages and data files by using digital signatures, encryption, compression and Radix-64 conversion. In addition, OpenPGP provides key management and certificate services. The GNU Privacy Guard (GnuPG) is the OpenPGP implementation of the GNU project. GnuPG is fully OpenPGP compliant, supports most of the optional features and provides some extra features. GnuPG is used as the standard encryption and signing tool of all GNU/Linux distributions.