Before I put Linux on the old PIII, it ran Win2kSP4 on 2.6 GB.
There's a bunch of things you can do before having to spend precious $$

Delete the windows rollback files that windows creates on install of the OS & on instal of every patch.
If you have any you will see the folders in the windows root eg:
C:\WINNT\$NtUninstallKB329115$
You can delete these rollback folders with no impact to your OS, unless you decide that you want to start uninstalling patches.

Also you can compress the drive:
right-click the drive icon in Windows Explorer, select properties, select the tick-box "Compress drive to save disk space". it is intelligent enough to leave frequently used files unpacked.

Uninstall any windows OS features and applications that you don't use.

Clear the Recycle Bin - Yes I know yours is empty, but check for any other users that have used that mahine under Documents and Settings. If you are admin you can clear their recycle bins. Remove any expired users if you can (note that deleting a user from the User management MMC pluggin may not delete their files from the machine) then delete any of their user folders that are left over.

Clean out the temp folder + temporary internet files (set cache days to 0 in IE and mozilla)

Clean out the temp folders in each users under the hidden "Local Settings" folder in the Documents and Settings folder ( a lot of stuff piles up in there)

Clean out the temp folder in the Windows root folder (C:\Windows\temp or C:\WINNT\temp)

Get into the Event Viewer and clear out all the recorded events (I have seen those get upto 500 MB over time)

See if you can reduce the size of your swap-file - set to a fixed size that suits that machine.

Do a full chkdsk /f and reboot. this forces the OS to look for lost chains etc and does a general drive & FAT file fixup.

Defrag after all that of course.

That should extend the life of the drive somewhat.

If the machine remains networked, you can map to another drive on another machine and keep the DC runtime files there. Almost all of them allow install to a different location other than the default.

Otherwise I think the USB drive is a neat solution because you can pack it into the Laptop case and take it with you.