Tuesday, October 11, 2016

The Anniversary Box

Today is our two-year wedding anniversary!  At our wedding, we asked our guests to write a note on a card that we would then open up ostensibly on our first anniversary.  Unfortunately, we were on vacation at the time and didn't get a chance to and decided to wait for our next anniversary to open the box.

I thought a lot of the notes were insightful, funny, or just cute, and so I transcribed some favorite quotes here!  If you recognize yours, and would like to be credited, just let me know and I'll de-anonymize you.


The sweet compliments (I think!)

"Everything is the best."

"I'm so lucky to have known you guys during this tough time of intern year.  I'm sure we'll feel next year is even worse."

"Hold hands--we've always noticed you look better than any other couple doing that."

"One thing that won't change is that you'll both still be amazing people with a great sense of fun and impeccably destructive logic."


I think that last one was a compliment, at least.

The sage advice

"Personal space can be awesome."

"Fly a quadcopter somewhere romantic."

"Your spouse is like a good cup of coffee--you may have it every morning but you never stop enjoying it!"

I wish I never stopped enjoying it.  It runs out so quickly!

Our quirky friends

"I don't have chocolate on me."

"Aren't you glad I don't draw for a living?"

"Our minds were blown at Costco today."

I really have to know.  What was it that blew their minds at Costco?

Wow

"Once upon a time, we were all hanging out at the Morton Arboretum, celebrating your love when all of a sudden a squirrel invaded your reception.  It jumped up on the bar and sat down next to the bottle of Disaronno.  It then promptly began to pour drinks for everyone.  He said, 'I want to embody the generosity that Ben and Simiao show to one another.'  The squirrel then proceeded to pass out drinks to all of the guests.  They were very excited and amazed by this talking, bar tending, love celebrating squirrel.  As the drinks settled in, the talented squirrel recognized it was time to take the celebration to the next level.  He promptly had the DJ start the music and broke out a soul train dance line.  The winning move was a specially designed move for the happy new couple.  It involved pins and ended with a fantastic split.  Sadly, a rabid dog ate the squirrel; However, a giraffe emerged from the woodwork and with one fell swoop won the hearts and minds of all attendees!  They stood in accord and said, 'Giraffe, you are excellent, talented, amazing and kind, but Ben and Simiao are far, far better."

... I don't even know what to say about that one!  I guess our drinks were pretty great!

Saturday, September 24, 2016

Photos and Anti-Photos

Simiao holds an inverse egg at the narrows
at Zion national park.
We don't give taking a photograph much thought these days.  We take our cell phone out of our pocket, point it at what we want to remember, and press a button.  We don't think about what's going on behind the scenes:  light hits the charge-counting device (CCD) for some amount of time, then at the end the image equals how many photons hit the charge-counting device over that period of time.  The world doesn't actually stand still for the CCD, though.  Time still passes, things move, life goes on.  The CCD, therefore, is actually reporting an average of what it saw while it was counting.  This can be apparent through motion blur or ruined photos if the photographer happened to be moving quickly while taking the picture.  When one takes a typical long exposure, this becomes even more pronounced because the transient things tend to disappear:  you get a picture of a city street with no cars, or a sidewalk with no pedestrians because the sidewalk and the street spent more time in the frame than the people or the cars.

What if we were able to take a picture of just the things that changed, and not the things that didn't?  You can perhaps imagine a landscape where you can see the clouds going by but not the mountains, where you could see the river going by but not the shore, where you could see the leaves blowing in the wind but not the tree trunks, and where you could see the people playing soccer but not the field.

This is what I call an anti-photo.

Part 1:  Solargraphs and Time-Lapse Video


This entire process can be done with a camera using a timer, but at the start of this project I actually just wanted something to do with old Android cell phones.  It turns out that an old cell phone is actually a great platform for taking long exposures and time lapses:  it's basically disposable because I wasn't doing anything with it anyway, it has a decent camera and amount of storage space, and is easy to set up as a time-lapse.

In order to take time lapses with an Android cell phone, I installed the excellent, free program Time and Tide - Lapse for Camera.  It can be configured to take a photograph at a configurable time interval, and gives you some essential controls over the camera settings:  for instance, if you intend on making a synthetic solargraph or anti-photo, you will probably want to turn off flash.  You will also want to set the focus to infinity, because otherwise the frame will subtly change over the time-lapse and lead to a blurry finished product once you composite the images.  It may also be worthwhile to set the ISO to 100 or otherwise something constant so that the same brightness on the image always corresponds to the same brightness of light.

Finally, simply set your camera up somewhere that it won't get bumped and let Time and Tide - Lapse run.  Once you have taken the desired series of photos, transfer them to your computer and the fun can begin!  It is straightforward to composite these images into a video in Linux using, for instance:

avconv -i %08d.JPG output.mkv





Next I wanted to make synthetic solargraphs from these collections of short exposures.  The solargraphs I made in previous posts are different from simply averaging the pixels in each photo in the series:  if I were to average each pixel, then a bright pixel in one photo would eventually be darkened if it was dark in subsequent photos.  In a solargraph, a pixel, once exposed, will never darken, only brighten with repeated exposure.  Thus, I wrote a special piece of code to sum up each pixel and then normalize the result to the brightest pixel in the series.  Here, for example, is a solargraph composite of the final video shown above:

A synthetic solargraph taken out the window of the old Google Chicago office.  Note how it has the tendency to sharpen the otherwise low-quality images coming from the repurposed Android phone.

The program that generated these images is provided for your utility.  It turns out that it is actually the same program that generates the anti-photos in the next section!

This is the point where the original intent of this post went awry.  It turns out that the re-purposed Android phones were great for doing traditional long exposures by taking many pictures and then integrating them with my solargraph program.  However, anti-photos were a bit trickier:  not only were the Android phones' cameras very noisy compared to a modern DSLR or electronic viewfinder camera, but the compression noise in the JPEG images they took made their images exhibit noise down to the pixel which made the resulting anti-photos unacceptable.

Part 2:  Anti-photos


It's now relevant to discuss why the same program generates both solargraphs and anti-photos, and how these two things are actually similar in some ways.

In a solargraph, every single pixel in a series of photos has its value added to that of the others, then a normalization happens at the end.  In an anti-photo, we only take into account pixels that have changed sufficiently from the previous photo; these are added to a running total, then normalized at the end.  The definition of "changed sufficiently" is left open to interpretation.  In an anti-photo, the definition of "sufficient change" is selected to be high enough so that in consecutive photos we throw out unchanged parts of the image.  If "sufficient change" was set high enough, then every picture would come out black because no pixel could ever change enough to trigger accepting it.  Generally, though, the higher "sufficient change" is defined to be, the darker the image will become.

Interestingly, if the definition of "changed sufficiently" is set to "no change", an anti-photo actually becomes a solargraph because every pixel is always included!  So, the same program is actually capable of doing both solargraphs and anti-photos.

Let's look at some examples!
A very early version of an antiphoto--I like that you can see the people walking down the sidewalk and the cars driving by, but there were a few issues with this series as well.  The compression noise at the edges of objects means that buildings are much more pronounced than they would be if I had shot in RAW mode.  The frame changed vastly in brightness over time besides, so you can see bright images in the reflections on the buildings.  Further, the intended subject--the people and the cars--were not well-centered in the shot.

Melting ice in three different glasses, illuminated by candelight.  You can see the shape of the surface of the ice cubes as they shrank and melted.
Little fish swimming around in a stream.  The fish are dark-colored, so taking a series of anti-photos revealed the bright stream bed rather than the dark fish.

The Chicago skyline in broad daylight.  During the series of photos, a cloud obscured the sun.  So, only the parts of the city directly illuminated by the sun (as opposed to illuminated by reflected light) survived the anti-photo procedure.  The net result?  Using the sun as the world's largest flash-bulb.

So, how do you take a good anti-photo, and how can you make your own?  The basic technique is simple.  Set your camera up on a tripod with a timer, then set the timer to take a picture periodically--perhaps every minute in the case of melting ice, or every second in the case of swimming fish.  Choose an interval that makes sense based on the rate at which your subject is moving.  Set the camera to have a constant shutter speed, a constant ISO, and set it to manual focus so that you always get the same frame and illumination.  Be sure to shoot in RAW mode, as the compression noise from lossy encoding like JPEG will add a significant amount of unwanted noise to the anti-photo algorithm.  As few as 30 photos can be assembled to make a great anti-photo.

The above techniques will go a long way towards taking an appropriate series of photos to make a good anti-photo, but I picked up a few extra tips from repeated practice:
  • Be sure to select a scene that, as much as possible, is uniformly illuminated over the series of photos.  Otherwise, you will pick up the changes in illumination as opposed to the change in subject.
  • Landscapes don't tend to make great anti-photos (the skyline above being a rare exception).  Try to select a subject that is the only thing changing in the frame, such as the fish or the ice.
  • The normal rules of framing a photo don't make sense for anti-photos.  Try to think about how the subject is moving and frame that motion, not the scene at large.  

Using my program to make an anti-photo is straightforward.  See the GitHub repository for a download link and instructions!

If you're into photography, I hope you will decide to try your hand and your eye at seeing the world's motion instead of seeing the world as a static image and make your own anti-photos.  When you do, please let me know what you come up with--I'll be happy to add some links to your work here!

Sunday, April 24, 2016

Ben Li-Sauerwine 2016


My good friend Chris Gaiteri made this awesome photoshop for me, which obviously needs to be shared with the world.  Vote Ben in 2016!

Sunday, April 10, 2016

Goodbye, Google

Friday was my final day as a software engineer at Google.  It's been an exciting 3 years!  I discovered a bug so big that it made the news.  I wrote code that ground over multiple terabytes of data in parallel to help with business decisions.  I helped with the Google Code Jam competitions.  I even wrote internal services that monitored our internal code quality to help with keeping our codebase clean and manageable.  I launched brand-new products and made meaningful improvements to old ones.  Most importantly,  I worked with wonderful colleagues in an environment where I felt appreciated and respected and where I learned to be a better coder every day.

Google isn't by any means a perfect company, but the internal culture is such that one can be candid and open about problems and as such it is constantly improving.  I think that this is not only a lesson that other companies could stand to take an example from, but an example that could be followed in one's personal relationships as well.

So, why did I leave?  After 3 years, I felt that it was time for a new challenge.  I've joined a startup as a partner with two former colleagues.  I'll be making strategic and hiring decisions as well as design and technical decisions and working directly with customers and investors.  Overall, it's a chance to develop new skills besides just my technical ones, and that's pretty exciting!  You'll be hearing more about it soon (there's no attractive website yet,) but you can check out my newly updated CV if you want a hint!

Monday, April 4, 2016

French Polynesia Stories


Last week Simiao and I were on vacation in French Polynesia.  Ever since I first saw a picture of the ring-shaped islands in the Pacific, I'd always wanted to visit.  Is the water really so electric blue?  Are the little islets really so close together that I could swim from place to place?  Are there really coconuts all over the place that I can just pick up, crack open and drink the water from?

The answer to all of these questions is a resounding "yes."  We spent 2 afternoons in Papeete, 2 days in Moorea, and 4 days in Bora Bora.  All of the locations were charming, but we preferred the natural beauty of Moorea and Bora Bora to the city in Papeete.  Initially I'd wanted to stay in the over-water bungalows, but unfortunately those were all spoken for 3 months in advance so we stayed in garden units instead.  In retrospect, not only were the garden units less expensive per night, but the 30-second walk to the lagoon was so convenient that I think I'd stay in the garden units next time I visit too.  What's more, in one case, the garden unit included a private plunge pool which I think is a nicer feature than ocean adjacency.

A Lemon Shark on a snorkeling tour we booked in Bora Bora.  The black fish are triggerfish.  These sharks have attacked divers in rare occasions, but don't normally pose a threat to humans.
Mt. Rotui on Moorea appears on the French Polynesian 50 and 100 Franc coins.  The mountain is a former volcanic peak, and the ring-shaped reef around the island marks the original maximum extent of the volcano.  The dormant volcano is slowly eroding into the ocean, but the ring-shaped reef keeps growing and will ultimately be the only remnant in the distant future.
Spinner dolphins on Dr. Poole's dolphin tour.  Intriguingly, while our hotel donated to his research it did not advertise or help us book his tour:  the hotel gets a kickback for booking the penned dolphin experience at the Intercontinental.  I highly recommend contributing to his research by patronizing his tour.  Fun and educational, we found Dr. Poole's tour to be one of the highlights of our trip.
The welcome party at the Sofitel Private Island on Bora Bora.  The best feature of the private island is that both the best reef for snorkeling and the best place to see rays in the lagoon are actually easily within swimming distance using flippers from the island, though weaker swimmers might prefer booking a tour.

We saw no fewer than five octopi in Bora Bora.  Incredibly, they could change their patterns and shapes on demand to suit their camouflage needs.  They could change from a dark red to blue-green and would alter the white spots to look like whatever rock they were hiding on. 

Monday, February 1, 2016

How We Beat Motion Sickness in The Witness

When The Witness was finally released, I was really excited that Simiao and I would be able to enjoy a video game together.  Previously, the only games that Simiao could really get into were old-school arcade and Atari games, but I thought that this was right up her alley:  puzzle focused, not scary, and I trusted that Jonathan Blow would settle for nothing less than brilliant design in his games.

Much to my dismay, we quickly learned that like many others Simiao experiences motion sickness in first-person style games.  It took several sessions of experimentation to come up with a permutation of settings that worked to make the game playable for her.  In case you find yourself in this position, here's what we had to do to make the game playable for her:


  1. Play the game with the lights in the room on.
  2. Position the monitor so that she is sitting a bit further back from it.
  3. Set the field of view to 90 degrees.  While there is no field of view slider in The Witness, this was attainable by right-clicking The Witness in steam, going to Properties then Local Files, clicking Browse Local Files, then opening Local.variables in the data folder.  Under ":/misc", we added the line "fov_vertical 90" to achieve this.
  4. My monitor, the ASUS PG278Q, has a GamePlus button on it.  By pressing this button on the monitor, I am able to overlay a crosshair on the center of the screen in the monitor settings.  This unchanging, central focal point helped with her motion sickness.
  5. Finally, we used the amazing RivaTuner Statistics Server to cap the frame rate at 30Hz.  It installs in a snap, appears to be fully compatible with GSYNC and The Witness, and offers a simple means to control frame rate.  We set the Framerate Limit to 30Hz and it worked like a charm.

We did try some chemical methods without success.  Simiao tried taking Benadryl which works similarly to Dramamine for motion sickness, and we tried ginger tea to settle her stomach.  Though these were not helpful, the ginger tea was delicious.  I couldn't help to think while discovering the techniques that did and did not work for Simiao that games should include a "motion sickness friendly" graphics mode that would automatically apply some of these settings to a game.  It would be fairly trivial to overlay a cross hair, increase the field of view, and limit the frame rate to something more friendly to those affected by motion sickness.

Together, the five techniques listed above enable us to enjoy The Witness together, which is a beautiful and intellectually satisfying game to play.  Now all we need is some more free time to play it in!


Saturday, November 7, 2015

Yellowstone Stories

We saw more elk around human habitats
in Yellowstone than in the wild.
For our one-year anniversary, Simiao and I went hiking in Yellowstone.  I'd always wanted to visit this national park, and I've never seen so many alien landscapes in such close proximity to each other:  thermal features, beautiful canyons and waterfalls, wildlife, petrified trees, and black obsidian beaches. If you decide to go to Yellowstone, I highly recommend that you go in the first week of the off-season.  Lower prices notwithstanding, the size of the parking lots gave me an idea of what a zoo the park must be during the summer with traffic jams, impossible parking and screaming children.  When we went in the first week of October it felt like we had the whole park to ourselves.

These are a few of my favorite photos.  I do have more related to an awesome side project I've been working on for almost a year now, but it's still not ready and needs some more work.  It'll be awesome when it's ready, trust me!

A toothpick-like tree at the Grand Prismatic Spring.
Watching this spooting thermal feature at the Artist's Paint Pots was like watching a natural lava lamp.
A squirrel fattening up for the coming winter.
The obsidian beach at Yellowstone Lake, the world's highest alpine lake.
A trio of fluffy gray jays were stalking us around Yellowstone Lake.
A fiery sunset over Great Fountain Geyser, which declined to erupt for the occasion.
A tree grows on top of a petrified tree stump on an incredibly steep trail on Specimen Ridge.
A view of the Lamar Valley from Specimen Ridge.