Dr. Lawlor's Code, Robots, & Things

August 10, 2018

Navier-Stokes GPU Fluids via Multigrid

Filed under: Graphics, Web — Dr. Lawlor @ 10:23 am

Check out my live demo of GPU fluids via multigrid.  It’s swirly!  I still need to write up the paper describing the multigrid pressure-free update, but here’s a 2015 lecture note describing the general approach.

(I’m reposting some of my old demos so I can show them off during SIGGRAPH 2018)


June 24, 2018

Legit Websites that Track Everything You Do

Filed under: Mass Surveillance, Random Thoughts — Dr. Lawlor @ 9:44 pm

TL;DR Version: many legitimate websites record extremely detailed data about everything you do on the site, including your mouse position and where you scroll, using a tool like ForeSee Replay.  Since there is no global opt-out, you need to install an ad blocker to prevent this kind of bandwidth-wasting privacy intrusion.

This afternoon I opened a few eBay tabs, and I have a vague recollection of seeing one of those standard ForeSee popups asking if I’d like to participate in a survey.  I almost always say “No Thanks”, and I’m 99% sure I did not agree to a survey today.

This evening, I noticed my laptop’s outbound network traffic was heavy, which seriously slows down our rural DSL (4mbit down, 1mbit up).  The more I looked at this, the less I liked it: chrome was sending piles of HTTPS data off to four different AWS-hosted servers that list themselves as “ForeSee Record Status (cxReplayRecorder) v2.4.11” (IP addresses:, but they’re load balanced).  I grabbed some of the traffic with Wireshark, but it’s encrypted, so I still have no way of telling what exactly was sent.

I knew the traffic was coming from chrome, but I have a bad habit of keeping about a hundred tabs open (!), so I carefully watched the bandwidth usage as I incrementally closed tabs.  As soon as I closed the ebay tabs, the traffic stopped.  Reopening the tabs didn’t bring the traffic back–it only happens when the tracker code decides it has enough data to be worth sending back out.

ForeSee seems particularly evasive in the way it phrases the ‘survey’: not only will you (maybe) answer questions, the ForeSee® Replay code also tracks every mouse click, mouse *hover*, and scroll that you do on the site.  Their Replay viewer lets them see heat maps showing where people click, or even hover the mouse.  (This is useful data for building the site, but it’s not disclosed that it is part of the ‘survey’.)

The tracking servers dump a pile of metadata, from which I can see:

  • There are about 10 active replay recorder servers right now.
  • Each server is receiving about 1,000 ‘transmits’ per minute.
  • Each ‘transmit’ occupies about 100 kilobytes (on average), which is a lot of bandwidth, and a lot of information captured.
  • A typical server seems to capture over a terabyte per week, from over 10 million users.

To stop this, install an ad blocker like the free Ad Block Plus, and add “/foresee/*” to the filter list.

May 24, 2018

Fusing basalt dust into obsidian glass using sunlight

Filed under: ISRU — Dr. Lawlor @ 7:59 pm

This is a public link to my old Facebook post about melting basalt dust into glass using my solar concentrator.   Plus a short newly released video of the melting process.

3D printed window pane aging well

Filed under: 3D Printing, ISRU — Dr. Lawlor @ 6:22 pm

The three-layer window pane I 3D printed last summer from clear PETG plastic has experienced zero issues so far, after a year of exposure to Fairbanks weather.  It doesn’t get much sunshine, but hasn’t leaked or noticeably frosted up.  And it still looks goofy!

April 14, 2018

Simple Encrypted Drive on Linux

Filed under: Linux, Random Thoughts — Dr. Lawlor @ 11:44 am

Here’s how to create a separate dedicated encrypted filesystem on a modern Linux machine.

  1. Create a large *sparse* file, here 60,000 MB maximum.  Sparse is faster to create and uses less storage, because the underlying file only gets filled in as you write data.  However, this may make the data write pattern visible to a sufficiently dedicated attacker; use “count=60000” from /dev/urandom instead of seek to prefill a dense file.
    • dd if=/dev/zero of=grades.img seek=60000 bs=1024k count=1
  2. Mount the file as an encrypted block device.
    • cryptsetup --cipher aes-xts-plain64 --key-size 256 create grades grades.img
  3. Make the filesystem.
    • mkfs.ext4 -m 0 /dev/mapper/grades
  4. Mount the filesystem to a directory.
    • mount /dev/mapper/grades grades
  5. You can now copy files on and off the encrypted path.  To clean up, unmount:
    • umount grades
  6. Now remove the block device and you’re secure again.
    • cryptsetup remove grades

April 2, 2018

Dissolving & Remelting Silver

Filed under: Chemistry, ISRU, Mining, Space — Dr. Lawlor @ 11:59 pm

My weekend side project these past few months has been doing chemistry on metals. I’ve been watching the Cody’s Lab precious metals refining series on YouTube, and the technology seems interesting and useful to me not only because I like working with metals, and live on a gold mine, but also for space exploration where we need to extract metals from rocks to be able to build things in space.

Yesterday I finished reprocessing a few dollars worth of sterling silver into fine silver, by dissolving the sterling in homemade nitric acid, precipitating out the silver by displacing it with copper, driving off the oxygen and other contaminants in my solar furnace, and finally melting a nice bead in a freshly made graphite crucible.

The full story and photos are on my Facebook post.  (There because it’s easier to post photos on Facebook.)


March 18, 2018

Martians, don’t let your babies grow up to be Idiots

Filed under: Random Thoughts, Space, Surviving Mars Game — Dr. Lawlor @ 11:35 pm

The simulated people in the game “Surviving Mars” have a bunch of character traits, including “Workaholic”, “Stoic”, etc.  One of the bad traits “Idiot”: when an Idiot is working in a building, they have a 10% chance of breaking the building bad enough to require maintenance (and critical spare parts).

You can choose who gets on the rockets, so I didn’t bring any Idiots to Mars.  Problem solved?

No.  After a few years, your colony celebrates its first birth.  If you don’t dedicate scarce dome space to a School and University, you’ll end up with a ton of Idiots in your first crop of Martian-born kids.  Since a fully-staffed semiconductor factory takes 10 people per shift, and an Idiot has a 10% chance per sol of breaking the building, if your workforce is all Idiots the factory basically never works, and your colony dies.

A cautionary tale!

(And a fun game, although I’m still trying to figure out how to get resources to reliably distribute themselves between depots as you expand.)

January 2, 2018

Migraine – Potassium – Sodium link?

Filed under: Random Thoughts — Dr. Lawlor @ 3:49 pm

Raw foods, like fruits, vegetables, nuts, and meats, include lots of potassium.  Modern processed foods include lots of sodium, because sodium is a cheap way to make foods tastier.

If you eat too many processed sodium-rich foods, your kidneys excrete the excess sodium, but this also washes potassium out of your body.

Yesterday, I skipped my usual daily apple and yogurt (potassium-rich foods), but I did eat a large helping of pesto pasta, which was green and tasty.  But then last night I woke up with a head-splitting migraine, including sensitivity to light and sound, which is quite unusual for me.  I also had an urge to eat something salty like corn chips: salty foods usually don’t work for me, but it indicates how your body reacts to an electrolyte imbalance.  Suspecting low potassium, I drank two glasses of apple-cherry 100% juice–each glass has 8% of your daily RDA of potassium, 1% of your RDA of sodium.  The migraine immediately began to recede.

Looking at the labels this morning, the processed pesto I used turns out to be high in sodium, with almost no potassium.

Disclaimer: I am a doctor, but of computer science, not medicine!  Actual medical doctors are now examining therapies that treat electrolyte imbalance to treat migraine.

November 15, 2017

“for int” considered harmful

Filed under: C++11, Programming — Dr. Lawlor @ 2:26 pm

In C or C++, the traditional variable to use in a loop is “int i”, typically something like:

for (int i=0;i<length;i++)  ... do stuff with i ...

Using “int” is traditional, but is a really bad idea, because “int” is only 32 bits on all modern machines.  This means any loop that should do more than 2 binary billion iterations will fail (binary billion == 1,073,741,824 == 1<<20).

If the loop iteration count exceeds what “int” can store, one option is a crash as int wraps around to -2147483648:

long length=3*1000*1000*1000L;
std::vector<char> v(length);
size_t count=0;
for (int i=0;i<length;i++) {
	if (0 == i%(128*1024*1024)) std::cout<<"i="<<i<<"\n";
return count;

(Try this in NetRun now!)

On my 64-bit Linux machine, this produces no compile warnings and works fine for lengths below 2 binary billion.  It all runs in about one second, including the time for std::vector to allocate and zero initialize 3GB of RAM.  But at runtime everything works fine until i overflows and wraps around:

Caught signal SIGSEGV

When i wraps around to -2147483648, the program is actually trying to write to array element v[-2147483648].  So worse than crashing, this introduces a potential security hole if an attacker can arrange for this index to point to valid memory.

If the program does not access memory, this acts as an infinite loop, since the int i can never reach the loop target.

In the program above, if we replace “long length” with “size_t length”, we at least get a compiler warning about comparison between signed (int) and unsigned (size_t) types.  But now the loop silently stops before finishing the correct number of iterations:

Program complete.

Here, i has wrapped around, but the compiler casts it to unsigned before comparing it with length: the loop is essentially converted to “for (int i=0;(size_t)i<length;i++)”.  Since casting a negative int i to unsigned size_t results in a huge value, the loop breaks out after 2 binary billion iterations, leaving the rest of the array untouched.

Needless to say, this will result in a very confusing data corruption bug.

Typecasting the comparison to work the opposite way, “for (int i=0;i<(int)length;i++)”, eliminates the warning but results in the long length converting to a negative int, and the loop executes zero iterations.

Hence using “int” as the loop index can result in:

  • A crash and/or security hole
  • An infinite loop
  • Skipping the last loop iterations
  • Skipping the loop entirely

Instead you should:

  • Prefer C++11 range-based for loops, since they’re clean and reliable
    • for (char &element : myVector)
  • If you can’t use range-based for, use size_t as a loop index (on 64-bit machines, size_t is 64 bits)
    • for (size_t i=0;i<length;i++)

If you’re stuck with int, you can’t reliably process data with more than 2 binary billion entries.  In a world where phones can have 8GB of RAM, people regularly process files exceeding 2GB, and your CPU can sling multiple gigs around in under a second, this is not a good idea!

November 8, 2017

Making chroot jails

Filed under: Linux, Sysadmin — Dr. Lawlor @ 8:20 pm

A chroot jail is a UNIX way run a dangerous application, like a network server, inside its own limited subset of the filesystem.  This cuts off access to recurring security holes like setuid executables in /bin or /usr/bin, kernel device files in places like /proc and /dev, and it lets you build your own restricted or sanitized runtime environment including libraries and config files.  Docker or rkt containers use chroot as one of the ways they sandbox containerized applications, and the fact all the libraries are included makes containers portable across systems.

A maximum security chroot jail consists of the prisoner process, and the bare minimum shared libraries necessary for it to run.  You can figure out which libraries are needed using “ldd ./prisoner”:

root@5a2c7cc2357f:/tmp/jailhouse# cp /bin/date ./prisoner
root@5a2c7cc2357f:/tmp/jailhouse# ldd ./prisoner
  linux-vdso.so.1 =>  (0x00007ffdf433d000)
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f39e8d18000)
  /lib64/ld-linux-x86-64.so.2 (0x000055751140c000)

(Or, since ldd is just a shell script wrapper around the dynamic linker library /lib64/ld-linux-x86-64.so.2, you can also run “/lib64/ld-linux-x86-64.so.2 –list ./prisoner”, although “ldd” is easier to remember.)

You can ignore linux-vdso, which is injected by the kernel, but if you copy the remaining shared libraries into the jail, you can run the prisoner process inside the jail using chroot.  Here, we just need ld-linux and libc:

root@5a2c7cc2357f:/tmp/jailhouse# mkdir lib64
root@5a2c7cc2357f:/tmp/jailhouse# cp /lib64/ld-linux-x86-64.so.2 lib64/
root@5a2c7cc2357f:/tmp/jailhouse# mkdir lib
root@5a2c7cc2357f:/tmp/jailhouse# mkdir lib/x86_64-linux-gnu
root@5a2c7cc2357f:/tmp/jailhouse# cp /lib/x86_64-linux-gnu/libc.so.6 lib/x86_64-linux-gnu/
root@5a2c7cc2357f:/tmp/jailhouse# chroot --userspec 12345:6789 . ./prisoner
Thu Nov 9 04:43:38 UTC 2017

Here we ran the prisoner as user ID 12345, in group ID 6789, due to the “–userspec 12345:6789” in the chroot call.  Without a userspec, the process runs in the jail as root, which is very bad (see chw00t)!  Neither this user nor group exist in my /etc/passwd or /etc/group files, which means the prisoner only gets a number, not a name.  You can add the username to either the system /etc/passwd or the chroot jailhouse/etc/passwd to get symbolic names, although do remember to set the permissions on jailhouse/etc/passwd carefully, because overwriting passwd with “jailbird:x:0:0::/home/jailbird:/bin/bash” could make the jailbird user run as root within the jail.

Often the process you want to run inside the jail is more complex, and so it needs quite a few libraries to work.  Sometimes those libraries load more files and libraries at runtime, and they’re not always well documented.  In these cases, “strace ./prisoner” is quite useful, because it shows you every system call, including calls like open (finding config files), exec (calling sub-programs), or mmap (usually loading shared libraries or allocating memory).  I often copy strace into the chroot so I can “chroot . strace ./prisoner” and watch what the program is doing inside the jail.

A particularly complex program may need /dev entries created (CUDA in a chroot needed basically all of /dev/nvidia*), or things like /proc mounted into the chroot using “mount -o bind /proc proc”.  These sorts of comforts make the jail more like a halfway house, in that it allows the prisoners more functionality but does pose more of a danger to society.

I’ve built chroot jails packed with ancient library versions so I can run ancient programs in their old environment.

I’ve also built chroot jails to contain student code (e.g., in NetRun).

Further reading:

Older Posts »

Blog at WordPress.com.