D-Light DTrace script for Sun Studio 12 in Solaris

Here's a pretty cool tool for developers, similar to the DTrace GUI from XCode in OS X 10.5 Leopard (Instruments):

It's part of Sun Studio 12.




It's still in development, but it can be pretty useful.

Sun Solaris DTrace - counting page-ins

Need more details than:

% kstat -n vm | grep pgin
anonpgin 727
execpgin 0
fspgin 1012885
pgin 999385
pgpgin 1013612
anonpgin 1347
execpgin 0
fspgin 187550
pgin 46875
pgpgin 188897

22:41 <> hello, is there a way to see what processes could be causing a page-in excess?
22:43 <> dtrace pgin

As always, DTrace to the rescue :-).

cthulhu# dtrace -n pgin'{@[pid, execname] = count();}'
dtrace: description 'pgin' matched 1 probe

^C

27462 gnome-keyring-da 1
27495 wnck-applet 1
430 dbus-daemon 2
27446 gnome-session 2
27483 nautilus 4
27319 Xorg 11

Staring SQL Developer or JDeveloper fails due to hidden messagebox

Can you spot why Oracle SQL Developer and JDeveloper aren't starting in this scenario?

You can't even see it on a white background (and there is no window in the taskbar) but there is a question box right behind the slash screen :-). Pay attention to the right edge of the splash...



Still, I love the new Java look.

C Programming under Tru64 UNIX

A crash course in C programming under Tru64 UNIX :-). You've got a text editor - ed(1), of course, make(1), install(1) and a C compiler - cc(1) - and you're good to go.

Don't like ED? Well, Tru64 also comes with vi and XEmacs by default, so no worries.



All you need to do now is write a Makefile:

% ed Makefile
a
CC=cc

hello: hello.o
$(CC) -o $@ $<

hello.o: hello.c
$(CC) -c -o $@ $<

clean:
rm -f hello hello.o
.
w
q

% make
cc -c -o hello.o hello.c
cc -o hello hello.c
% ls
Makefile hello hello.c
% hello
Hello, Tru64 UNIX!
% make clean
rm -f hello hello.o
% ls
Makefile hello.c

IBM z/OS - OS/390 UNIX on zSeries Mainframe Development - C89

Here's a nice quick intro to C programming on IBM mainframes :-).

The C Programming Language

To program using the C Programming Language on OpenVMS, you need the DEC C compiler.

$ run CC073.DCX_AXPEXE;1
$ @SYS$UPDATE:VMSINSTAL
* Products: *

Then we test it:

SET TERMINAL/DEVICE=VT102
EDIT HELLO.C

#include "stdio.h"
int main()
{
printf("Hello, OpenVMS CC!");

return (0);
}

[End of file]


$ CC HELLO.C
$ LINK HELLO.OBJ
$ RUN HELLO.EXE
Hello, OpenVMS CC!


C++ is similar, but you need CXX instead of CC.

Installing Java 1.5 JDK on OpenVMS 8.3 Alpha

I've already covered ancient version of Java and OpenVMS, now it's time for something more recent. Just grab the latest Java 1.5 JDK for OpenVMS from HP.

$ RUN DEC-AXPVMS-JAVA150-V0105-3-1.PCSI_SFX_AXPEXE

$ PRODUCT INSTALL JAVA150 /SOURCE=SYS$SYSROOT:

[SYSMGR] /NORECOVERY_MODE

$ PRODUCT EXTRACT FILE JAVA150 /SOURCE=SYS$SYSROOT:

[SYSMGR] /SELECT=RELEASE_NOTES.HTML /DEST=[]

$ @JAVA$150_SETUP.COM;1

$ java -version
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition
Classic VM (build 1.5.0-3, 03/01/2007-14:39, native
threads, jit)

$ javac -version
javac 1.5.0

$ SET TERMINAL/DEVICE=VT102

$ EDIT hello.java
class HelloWorldApp {
public static void main(String[] args) {
System.out.println("Hello, OpenVMS!");
}
}
^z

$ javac hello.java

$ java hello
Hello, OpenVMS!

Java Development on OpenVMS

Got an OpenVMS box and you need to run some Java stuff?

$ SHOW SYSTEM
OpenVMS V7.3 on node VMS 4-MAR-2008 22:26:36.67 Uptime 0 01:16:05

$ JAVA -version
vms$dka0:[sys0.syscommon.][sysexe]java$java.exe;1 version "1.1.8-5"

Well, if you happen to have an older version of OpenVMS and Java (you can download a newer JDK - 1.5 here) you're going to have to set it up properly first. Set the CLASSPATH too.

$ @SYS$MANAGER:JAVA$SETUP.COM
Setting up symbols for foreign command line usage...
%DCL-I-SUPERSEDE, previous value of JAVA_COMPILER has been superseded
%DCL-I-SUPERSEDE, previous value of JAVA$FILENAME_CONTROLS has been superseded
JAVA$FILENAME_CONTROLS now set to: -1

$ DEF JAVA$CLASSPATH "/sys$common/java/lib/JDK118_CLASSES.ZIP:."
%DCL-I-SUPERSEDE, previous value of JAVA$CLASSPATH has been superseded

Great, now that we've setup our JAVA environment, we can start developing applications :P.

$ ED HW.JAVA
class hw
{
public static void main(String args[])
{
System.out.println("Hello, OpenVMS!");
}
}

^Z (Type Control Z to save and exit)

$ TYPE HW.JAVA
class hw
{
public static void main(String args[])
{
System.out.println("Hello, OpenVMS!");
}
}

$ JAVAC HW.JAVA

$ JAVA HW
Hello, OpenVMS!

Enjoy your ancient version of Java...

PS: Once you've had your fun, just download JDK 1.5 :-).

PowerShell, PowerTab, PSCX, Console, Terminus and Vim

Here's something for all you Powershell / WMIC / VB / cmd script monkeys :-).

You can, of course, script in ksh, Perl or Python if that's your thing. (ActiveState Perl / Python).

Powershell + PowerTab + PSCX + Console + Terminus Programmers Fonts + VIM + VIM PowerShell Syntax Highlighting.



Here's another PowerTab Theme and PSCX (PowerShell Community Extensions):



A cyan BackColor to match my prompt:



And of course, a good color scheme and PoSH syntax highlighting in VIM is also nice:

Programmer Fonts and VIM Themes

A great collection of monospaced / fixed width programmer's fonts.

My favorite font is Terminus. It goes great with the "Slate" VIM theme.

Windows Performance Tracing Toolkit

Windows Performance Tools Kit, v.4.1.1 helps diagnostic application start time issues, boot issues, deferred procedure calls and interrupt activity (DPCs and ISRs), interrupt storms, application resource usage and system responsiveness issues.

The toolkit includes xperf - a trace capture tool, xperfview - a visualization tool (Performance Analyzer) and xbootmgr - a boot trace capture tool.

It works great along side sysinternals tools (Process Explorer and Process Monitor) and krview for kernel tracing and profiling and Performance Monitor (perfmon.msc).


Here's a VERY simple trace (xperf -on DiagEasy, xperf -d trace.etl, xperf trace.etl). There's hunderds of knobs you can turn. You can use it for everything from getting VERY detailed system information (xperf –i trace.etl –a sysconfig) to getting advanced disk I/O info or pinpointing Registry Access bottlenecks.

MySQL, Oracle, DB2, PgSQL and Firebird versus Leap Years and Division by Zero

Oracle first:

SQL> CREATE TABLE leaptest (thedate date);

Table created.

SQL> INSERT INTO leaptest VALUES ('28-feb-2008');

1 row created.

SQL> INSERT INTO leaptest VALUES ('29-feb-2008');

1 row created.

SQL> INSERT INTO leaptest VALUES ('30-feb-2008');
INSERT INTO leaptest VALUES ('30-feb-2008')
*
ERROR at line 1:
ORA-01830: date format picture ends before converting entire input string


SQL> INSERT INTO leaptest VALUES ('29-feb-2007');
INSERT INTO leaptest VALUES ('29-feb-2007')
*
ERROR at line 1:
ORA-01830: date format picture ends before converting entire input string

SQL> SELECT * from leaptest;

THEDATE
---------
28-FEB-08
29-FEB-08

As you can see, Oracle knows 2008 is a leap year, and that 2007 is not.

Let's try another good database, PostgreSQL:

goods=> CREATE TABLE leaptest (thedate date);
CREATE TABLE
goods=> INSERT INTO leaptest VALUES ('28-feb-2008');
INSERT 0 1
goods=> INSERT INTO leaptest VALUES ('29-feb-2008');
INSERT 0 1
goods=> INSERT INTO leaptest VALUES ('30-feb-2008');
ERROR: date/time field value out of range: "30-feb-2008"
goods=> INSERT INTO leaptest VALUES ('29-feb-2007');
ERROR: date/time field value out of range: "29-feb-2007"
goods=> SELECT * FROM leaptest;
thedate
------------
2008-02-28
2008-02-29
(2 rows)

No problems here either. What about IBM DB2?

db2 => CREATE TABLE leaptest (thedate date)
DB20000I The SQL command completed successfully.
db2 => INSERT INTO cmihai.leaptest VALUES ('2008-02-28')
DB20000I The SQL command completed successfully.
db2 => INSERT INTO cmihai.leaptest VALUES ('2008-02-29')
DB20000I The SQL command completed successfully.
db2 => INSERT INTO cmihai.leaptest VALUES ('2008-02-30')
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0181N The string representation of a datetime value is out of range.
SQLSTATE=22007
db2 => INSERT INTO cmihai.leaptest VALUES ('2007-02-29')
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0181N The string representation of a datetime value is out of range.
SQLSTATE=22007
db2 => SELECT * FROM cmihai.leaptest

THEDATE
----------
02/28/2008
02/29/2008

2 record(s) selected.

Let's even try Firebird:

SQL> CREATE TABLE leaptest (thedate date);
SQL> INSERT INTO leaptest VALUES ('28-feb-2008');
SQL> INSERT INTO leaptest VALUES ('29-feb-2008');
SQL> INSERT INTO leaptest VALUES ('30-feb-2008');
Statement failed, SQLCODE = -413
conversion error from string "30-feb-2008"
SQL> INSERT INTO leaptest VALUES ('29-feb-2007');
Statement failed, SQLCODE = -413
conversion error from string "29-feb-2007"
SQL> SELECT * FROM leaptest;

THEDATE
===========
2008-02-28
2008-02-29

Well now, let's try MySQL:

mysql> CREATE TABLE leaptest (thedate date);
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO leaptest VALUES ('28-feb-2008');
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> INSERT INTO leaptest VALUES ('29-feb-2008');
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> INSERT INTO leaptest VALUES ('30-feb-2008');
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> INSERT INTO leaptest VALUES ('29-feb-2007');
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> SELECT * FROM leaptest;
+------------+
| thedate |
+------------+
| 0000-00-00 |
| 0000-00-00 |
| 0000-00-00 |
| 0000-00-00 |
+------------+
4 rows in set (0.00 sec)

As you can see, MySQL happily takes the input, but when we try to read it.. surprise surprise, your data isn't there! It should respond with an error code..

As Komal Shah points out, that's only due to an invalid date format, using ISO dates dates works fine.

INSERT INTO leaptest VALUES ('2008-02-28');

An invalid date will still generate a warning and a 0000-00-00 entry.

Here's another fine piece of MySQL behavior:
Division by 0:
Well, let's first see how Oracle and PgSQL handle this:

Oracle

SQL> SELECT 0/0 FROM dual;
SELECT 0/0 FROM dual
*
ERROR at line 1:
ORA-01476: divisor is equal to zero

PostgreSQL

goods=> SELECT 0/0;
ERROR: division by zero

IBM DB2:

db2 => SELECT 0/0 FROM cmihai.leaptest
SQL0801N Division by zero was attempted. SQLSTATE=22012

Firebird:

SQL> SELECT 0/0 FROM leaptest;

=====================
Statement failed, SQLCODE = -802
arithmetic exception, numeric overflow, or string truncation
SQL> SELECT 0/1 FROM leaptest;


=====================
0
0
SQL> SELECT 1+1 FROM leaptest;


=====================
2
2

MySQL
mysql> SELECT 0/0;
+------+
| 0/0 |
+------+
| NULL |
+------+
1 row in set (0.00 sec)

So there you have it, MySQL unexpected behavior.

If anyone cares, tested versions: DB2 9.5, Oracle 10g and 11g, PosgreSQL 8.2, 8.3 and Firebird 2. MySQL was 5.0.45.

Getting started with PostgreSQL - a Database Primer

A quick guide to getting started with the PostgreSQL database server:

Download and install PostgreSQL. If you're using some kind of BSD or Linux, it's probably already in the ports / packages repository. You may need to use initdb to create a new database cluster, and edit pg_hba.conf to grant your users / machines connect privileges to the server. If you're in a UNIX-like OS, you'll need to "su - postgres". After that:

Connect as postgres:

psql -U postgres

Create a role:

CREATE ROLE testuser LOGIN PASSWORD 'test123';

Create a database:

CREATE DATABASE testdb;

Note that the database is created in the cluster ENCODING. If you want to specify a different encoding (not recommended) you can use something like:

CREATE DATABASE testdb OWNER = testuser ENCODING = 'UTF8';

List databases, groups, users, quit:

\l+ \dg+ \du+ \q

Connect as the new user to the new database:

psql -U testuser testdb

Create a table:

CREATE TABLE test (ID serial PRIMARY KEY, name varchar(25) NOT NULL UNIQUE);

List table, relations, schemas, describe table:

\d

\dt

\dn

\d test

Insert some data:

INSER INTO test VALUES (default, 'quux');

Do a query:

SELECT * FROM test;

\q

Delete a database:

psql -U postgres

DROP DATABASE testdb;

Once you've got the hang of things, you can use \? and \h for help:
\?
\h
\h CREATE DATABASE

You can now delete the testuser role:

psql -U postgres
DROP ROLE testuser;

Create your new username and database, and connect to it. Create a simple database schema.

Here's a fun PostgreSQL feature: HTML output. \H turns it on, \H again turns it of:

We have a very simple table with 3 entries:

goods=> \d countries;
Table "public.countries"
Column | Type | Modifiers
---------+-----------------------+-----------
country | character varying(50) | not null
Indexes:
"countries_pkey" PRIMARY KEY, btree (country)

goods=> SELECT * FROM countries;
country
---------
A
B
C
(3 rows)

\H
goods=> \H
Output format is html.
goods=> SELECT * FROM countries;

We can use \o to redirect the output to a file:

goods=> \o countries.html
goods=> SELECT * FROM countries;

\q and view the file in your web browser :-).



Now that you've got the feel of psql, you can use PgAdmin III (or PHPPgAdmin or whatever interface you like, like OpenOffice.Org Base or Microsoft Office Access). PgAdmin III also shows the SQL commands it's going to run, so it's a good way to learn SQL.




Use this for a tutorial and this for reference.

Using OpenOffice.org Base to access PostgreSQL databases

Just like Access can connect to a PostgreSQL database via ODBC or JDBC, so can OpenOffice.org Base. But the good news is we don't need to use ODBC or JDBC, as there is a native PostgreSQL SDBC driver for OpenOffice.

Download the postgresql-sdbc-ver.zip file, start OpenOffice (if you start writer type Ctrl-W to close it) so you can access Tools - Extension Manager. Add the downloaded driver, then restart OpenOffice (you also need to exit the QuickStarter. Just right click the SystemTray icon - Exit).



Now you can start OpenOffice.org Base and connect to a PostgreSQL database:



In the Connections Setting type: "dbname=YourDBName host=YourDBServerIP"



Setup a username and password:



That's pretty much it for the database setup, now you can start creating your database. You can use data already available in the PostgreSQL server, or create new tables. You then build Queries, Forms and Reports based on those tables.

You can use the Table Design or Table Wizard to create tables:



You can use the Relationship Editor to create Foreign Key constraints:



You can use the Form Design and Form Wizard to create Forms and the Report Design / Wizard for Reports:



It's usually a good idea to take a look with psql every now and then. OpenOffice.org Base isn't exactly perfect when it comes to creating the database schema, and especially when it comes to constraints. Of course, neither is the SDBG drivers, so it's best to consider this "just for fun" for the moment.

VisualLogParser - A LogParser GUI - Parsing logs using SQL syntax

LogParser is an awesome Windows application for parsing through system logs (event viewer, Registry, Active Drectory, IIS, etc) using a somewhat SQL-like syntax. It can also generate detailed reports, graphs, etc. Also makes a nice starting point for various Data Mining operations on your logs.

VisualLogParser is a GUI for LogParser. It is available on Microsoft's Open Source project page, Codeplex.

Here is the discussion board for Visual LogParser.

Why I don't trust software that puts 3rd party drivers in my system

Here's something fun I found on somebody's computer... it kept crashing.. I wonder why :-). Let's check what WinDbg !analyze -v has to say of the minidumps:

DEFAULT_BUCKET_ID: DRIVER_FAULT

Probably caused by : SYMTDI.SYS ( SYMTDI+ab3f )


STACK_TEXT:
WARNING: Frame IP not in any known module. Following frames may be wrong.
a74c1aec a73b874e ebe44589 863d4220 00000e20 0xd04d8dda
a74c1b1c a73b855d 86cd0034 a74c1b58 863d4220 SYMTDI+0x1174e
a74c1b60 a996bfbf 885f7700 863d4220 00000e20 SYMTDI+0x1155d
a74c1b64 885f7700 863d4220 00000e20 00000000 vsdatant+0x40fbf
a74c1b68 863d4220 00000e20 00000000 00000034 0x885f7700
a74c1b6c 00000000 00000000 00000034 a74c1b9c 0x863d4220

OK, here' s another one:

STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be wrong.
8a3e1a30 a7899aef 88c908b8 88bbef00 00000000 SYMTDI+0xab3f
8a3e1a50 a789c490 88c90404 88bbef00 00000000 SYMTDI+0xaaef
8a3e1a68 a78a99ab 88bbef00 00000000 8a3e1aac SYMTDI+0xd490
8a3e1a94 a78aadb3 89826688 8275b620 89826688 SYMTDI+0x1a9ab
8a3e1aa8 804ef095 89826688 8275b620 837bfb58 SYMTDI+0x1bdb3
8a3e1b10 a9ebc5e5 89429488 0103fd04 a9ebc5e5 nt!MiCheckControlArea+0x103
8a3e1c5c 8057f1fd 869af238 00000001 0103fc48 afd!AfdFastIoDeviceControl+0x415
8a3e1d00 805780c2 00000340 000002d4 00000000 nt!KeInitThread+0x101
8a3e1d34 8054086c 00000340 000002d4 00000000 nt!RtlCreateAcl+0x1d
8a3e1d64 7c90eb94 badb0d00 0103fc14 baccfd98 nt!RtlIpv4StringToAddressExA+0x149
8a3e1d78 00000000 00000000 00000000 00000000 0x7c90eb94

3: kd> lm kv
a788f000 a78b9dc0 SYMTDI T (no symbols)
Loaded symbol image file: SYMTDI.SYS
Image path: SYMTDI.SYS
Image name: SYMTDI.SYS
Timestamp: Sat Aug 24 23:54:56 2002 (3D6800B0)
CheckSum: 00038527
ImageSize: 0002ADC0
Translations: 0000.04b0 0000.04e0 0409.04b0 0409.04e0

No point bothering with Driver Verifier at this point, this is cleary caused by Norton Internet Security - Norton Dispatch Drivers. First few google hits confirm this:

Random BSOD's and reboots. - TechSpot Troubleshooting - Probably it is caused by SYMTDI.sys (Norton Internet Security Filter) or faulty memory

MSFN Forums > Server 2003 Blue Screen with SYMTDI.sys

Emacs.NET

"Imagine if someone wanted to write something like Emacs.Net. Actually don't imagine it, it is happening."

"We are looking for developers/testers to build a tool that I will roughly describe as "Emacs.Net"."

^^ duglasp at microsoft dot com

Well, this should be interesting to watch. Hell, notepad could use a rewrite :P.

HP Calculator Emulators for the PC - your very own HP-48

HP-48 was a graphing calculator that uses Reverse Polish notation using RPL (Reverse Polish LISP). It's pretty the best calculator. Ever.



http://www.hpcalc.org/hp48/pc/emulators/

The HP calculators are fully programmable, and you can also download a LOT of applications for it. Even games :-).

Core Java - 8th edition

The 8th edition of Core Java is out, one of the best books on practical Java programming. The source code is freely available for download.
http://horstmann.com/corejava.html

Exploit development frameworks and platforms - Metasploit, MSF-XB

Metasploit:

The Metasploit Project is an open source computer security project aids penetration testing activities and IDS signature development and provides information on security vulnerabilities.



Components:

• The Metasploit Framework is a development platform for creating security tools and exploits. The framework is used by network security professionals to perform penetration tests, system administrators to verify patch installations, product vendors to perform regression testing, and security researchers world-wide. The framework is written in the Ruby programming language (rewritten from Perl) and includes components written in C and ASM. The Metasploit Framework consists of tools, libraries, modules, and user interfaces. The basic function of the framework is a module launcher, allowing the user to configure an exploit module and launch it at a target system. If the exploit succeeds, the payload is executed on the target and the user is provided with a shell to interact with the payload.
• The Shellcode Archive contains various payloads written by the Metasploit staff. Has Windows Shellcode Development Kit.
• The Opcode Database contains the position of certain machine language opcodes in the attacked program or included DLLs
  

MSF-eXploit Builder

• MSF-eXploit Builder (MSF-XB) is free Windows GUI and Exploit Development PlatformMetasploit Framework exploit modules. It will help you to edit/modify/create/test exploit modules for the Metasploit Framework. It also contains an assortment of Fuzzers (TAOF, ProxyFuzz, FileFuzz, WinFuzz) and various other tools (Branchseeker, Faultmon, mycrc, nc, Findjmp2 and even pstools). It requires an installed Metasploit framework and a debugger (try Immunity Debugger).

SecurityForest Exploitation Framework:

• SecurityForest's Exploitation Framework is similar in concept to Metasploit, and is written in Perl. The major difference is that it leverages the massive amount of exploits available in the ExploitTree. These exploits are publically available and do not have to be re-written to be used in the framework (no matter what language and sometimes no matter what OS). It basically acts as a Graphical User Interface to the ExploitTree which is dynamically updated at the same time as the ExploitTree.

E-mail exploitation frameworks:

• PIRANA is an exploitation framework that tests the security of a E-mail content filter. By means of a vulnerability database, the content filter to be tested will be bombarded by various emails containing a malicious payload intended to compromise the computing platform. PIRANA's goal is to test whether or not any vulnerability exists on the content filtering platform. This tool uses the shellcode generator from the Metasploit framework!

Browser Exploitation Framework:

• BeEF is the browser exploitation framework used to demonstrate the real-time impact of XSS browser vulnerabilities. Download here.
• Nikto is an Open Source web server scanner which performs comprehensive tests against web servers for multiple items, including over 3500 potentially dangerous files/CGIs, versions on over 900 servers, and version specific problems on over 250 servers. Scan items and plugins are frequently updated and can be automatically updated (if desired).

Network Vulnerability Scanners

• Nessus is a comprehensive vulnerability scanning program. Its goal is to detect potential or confirmed weaknesses on the tested machines.
• FwTest is a firewall testing tool.

Online vulnerability databases:

• Secunia provides security advisories and information about patches, and provides software for vulnerability management.
• Milw0rm is an exploit database separated by exploit type.