Friday, August 24, 2012

How to Roll Back Packages in R

Today I was in an unusual position where changes to a common dependency in one R package broke compatibility with several other packages I use for predictive modeling.  When I went to roll back the package, I was surprised to learn that there is no option to install.packages which will simply install a version of my choice.  Instead, Matt taught me a rather simple and straightforward way to install an earlier version using Hadley's devtools package.  The first step is to install the package and load it:

install.packages("devtools")
library(devtools)


Next, I can look up the package and version of my choice on the CRAN archive.

Finally, once I find the package and version I want, I can copy the link from the CRAN archive and use.  For example,

install_url("http://cran.r-project.org/src/contrib/Archive/survival/survival_2.36-12.tar.gz")

Just like that, I've rolled back the package to an earlier version.  Thanks to Matt for introducing me Hadley's useful devtools package!

Sunday, August 19, 2012

The Yacht Race

I'd never sailed before this week, but my friend Matt invited me to a yacht race and then again sailing the next day this week and I'm sold.  There's nothing quite like the sensation of the wind pulling you through the water when you have the sails aligned just right, and the views of Seattle and Mount Rainier from the Puget Sound are really marvellous.  Let it be known:  a cooler full of beer, some friends, and a yacht are a formula for a great evening!  We were in such a hurry to get out the door that I forgot to bring my camera to the yacht race, so the pictures posted here are crude cell-phone photographs that don't quite do the day justice. 




The funny thing is, I think Matt actually appreciated the opportunity to just enjoy the ride and not drive for a change.  Keeping the boat aligned to catch the wind just right takes some getting used to, but just trying is a lot of fun!


Saturday, August 4, 2012

Playing Chicken with History

I watched with some interest the news regarding Chick-Fil-A Appreciation Day.  The concept is that because Chick-Fil-A as a company publicly opposes gay marriage, then you can show your opposition to the same by supporting them.  I wondered if this move could be less a function of the company owners' core beliefs and politics than a marketing scheme designed to drum up sales based on the general opinion of those that live near Chick-Fil-A stores.

I do my best to keep politics out of my blog and focus on science, but this time I feel the need to preface a study with what shouldn't be a politically charged statement.  Majority vote is a pretty good way to deal with mundane, day to day problems.  If most people believe that the patent system should be overhauled, or most people believe that a road should be built, then that should probably happen.  No majority vote, government, religion or individual, however, has the right to disenfranchise anybody of their civil or human rights.  Marriage is a human right, and states that have constitutionally banned gay marriage based on a referendum are wrong.  People who believe that gay marriage should be illegal are also wrong.  They are also on the losing side of history.  I personally will not eat at a Chick-Fil-A restaurant until they publicly renounce their support of bigotry.

As a scientist, what I'm mostly interested in as I read about Chick-Fil-A Appreciation Day is whether the Chick-Fil-A debacle is a marketing move, and whether we will see other businesses take controversial political positions in order to boost sales and brand awareness.

First, I need some data to work with.  I obtained data on the locations of Chick-Fil-A stores using a map of the geographic distribution from their company website.  That map, correct on June 1, 2012 is reproduced below.

Figure 1:  Geographical distribution of Chick-Fil-A stores on June 1, 2012.  This map is copyright CFA properties Inc., and is reproduced for non-profit educational purposes as permitted by section 107 of the copyright law (title 17, US Code.)

It is remarkably difficult to find data on the state-wise public opinion on gay marriage, so as a proxy I used raw data from the Pew Research Center's 2011 Political Typology Survey.  In particular, one question was "q37u:  Should homosexuality be accepted by society?", and I assume that one's answer to this question equals whether or not they support legal recognition of same-sex marriage.  Admittedly, it's possible that some people may see societal acceptance of homosexuality as something different than marriage equality, but it's the best proxy I could find.  If you have better data by state, please let me know!

The first step after downloading the SPSS file was to convert it to an open format that I could actually use.  I found GNU PSPP to be inadequate for this purpose, and instead used R to read a SPSS save file into a data frame:

library(foreign)
pew <- read.spss('~/Desktop/2011 Political Typology public.sav', to.data.frame=T)


Now I am able to break down the answers to this question by state.  Answers 1 and 2 to the question "q37u" felt that homosexuality should be accepted by society, and 3 and 4 said that homosexuality should be discouraged.  Those counts are then given by

accept <- table(pew[as.numeric(pew,"q37u") %in% c(1,2), "state")


and

reject <- table(pew[as.numeric(pew,"q37u") %in% c(3,4), "state")


Some states had very few data points (Wyoming had only 3), and Alaska and Hawaii were totally unrepresented.  As such, the error bars on the ratio of accepters versus discouragers can be quite large.  That said, producing a map of acceptance of homosexuality by state produced a very telling result.  With help from the Revolution Analytics Blog, then:

# gives proportion acceptance
acceptanceRatio <- accept/reject        

# To lower case
names(acceptanceRatio) <- tolower(names(acceptanceRatio))

# Ok, let's get the appropriate regions from the map
require(maps)

# Get the region names
regionNames <- map("state",namesonly=TRUE)

# Need to parse out the name, exactly
stateName <- unlist(lapply(as.list(regionNames), function(x) {return(strsplit(x, ":")[[1]][1])}))

# Set the ratios by state
stateRatios <- acceptanceRatio[stateName]


# Generally accepting, generally rejecting, about even. (60% majority)
color <- apply(stateRatios, 1, function(x) {
  if (x <= 2/3) {return(rgb(red=1, green=0, blue=0))}
  if (x >= 3/2) {return(rgb(red=0, green=1, blue=0))}
  return(rgb(red=1,green=1,blue=0))
})

map("state", fill=TRUE, col=color)


Figure 2:  Opinion on societal acceptance of homosexuality by state.  In green states, over 60% of respondents to Pew Research's 2011 Political Typology Survey supported societal acceptance of homosexuality.  In red states, over 60% of respondents felt that society should discourage homosexuality.  In yellow states, there was no 60% majority.  Please note that there is substantial margin of error here, as some states had as few as three respondents.

Figure 2 was, to me, an extremely telling plot.  First, I'll point out that there is obviously strong overlap between the number of Chick-Fil-A stores in a state (red in figure 1) and sentiment that the state should discourage homosexuality (red and yellow in figure 2).  What really shocked me, though, is that this plot tells me that a vocal minority is responsible for the outcry against same-sex marriage.  Even in North Carolina, where a constitutional amendment was voted into effect to ban gay marriage, the ratio of those who felt that society should accept versus discourage homosexuality is nearly 1:1.  With voter turnout below 50% in most of the country, this means that all that has to happen in order to win marriage equality in most states is for those who support equality to go out and vote.

So, was it a good idea for Chick-Fil-A to market their bigoted position?  Nationwide, no.  More people support equality than reject it.  In the regions of the country with the most Chick-Fil-A stores, it's a dicey measure as the population is mostly split.  The reality, though, is that a look at the national political scene shows that the vocal minority is much more active in promoting their historically doomed cause than the majority which supports acceptance.  In that sense, Chick-Fil-A's move is a pretty safe bet.  Gay rights supporters simply aren't as active as their opponents and may not care enough to give up fried chicken.  Meanwhile, Chick-Fil-A enjoys a surge of popularity from the vocal minority.

In a sense, Amazon quickly followed Chick-Fil-A's experiment with their own.  In Jeff Bezos' case, it makes sense to side with the majority of people when your company makes sales nationwide.  President Obama also recently took the side of the majority, a wise choice when running for office with a nationwide constituency.  It will be interesting to see whether or not we see more examples of high-ranking representatives of companies taking politically controversial positions to market to their primary customer base in the future.

In conclusion,

  • A majority of, or at least a roughly equal number of people, support societal acceptance of homosexuality most of the lower 48 states.
  • Proponents of inequality are apparently much more vocal and active in promoting their position than those who support equality.
  • Chick-Fil-A's position actually conflicts with that of about half of their potential customers even in the Southeast, making their move a dicey one at worst but a good one at best based on the activeness of the anti-gay rights crowd.

Finally,

  • "All that is necessary for the triumph of evil is that good men do nothing." -Edmund Burke

Get out and vote!  If you support equality, you're in good company.  In most states, you represent 60% or more of the constituency.  In nearly all states, you represent 40% or more of the constituency.  All that has to happen for equality to win the day is for you to tell the government what you think!





Sunday, July 22, 2012

Washington State Primary Ballot: Who Are They?

I had a new experience yesterday:  I got my primary election ballot in the mail!  Coming from Pennsylvania and having registered non-partisan, I was not allowed to vote in Pennsylvania primaries.  I always thought that was odd:  if you're a political party, wouldn't you care about what the people with no party affiliation thought about your candidates as much as or more than those in your party?  Since people in your party are not likely to change sides, the independents are the people most likely to be swayed to your side in an election!

Anyhow, I was dismayed when I saw the primary ballot for two reasons.  First, I saw that the ballot included such names as "Mike the Mover" and "Goodspaceguy."  To me, these non-names don't inspire much confidence in the candidates, and it turns out their websites didn't either.  Second, when I started searching for candidates, I noted that many didn't even have a website, or if they did their website didn't include meaningful information about their platform.  I don't want this to be a blog about anything but facts and opinion supported directly by facts, so I will keep this post simple.  What follows is simply a table of candidates on the Washington State Primary Election ballot, and if I could find a website why you should vote for them in their own words.  I hope you will use this information to make the best informed decision in the upcoming primary and election, and may the best candidate win!  Please also be aware of the election pamphlet available at the King County website, which I found helpful but incomplete.


For United States Senator
NamePartyWhy you should vote for
Michael BaumgartnerRepublicanvotebaumgartner.com
Will BakerReformNo Website Found
Chuck JacksonRepublicanscaryreality.com
Timmy (Doc) WilsonDemocratictimwilsonforsenate.org
Art CodayRepublicanartcoday.com
Maria CantwellDemocraticcantwell.senate.gov
Glen (Stocky) R. StockwellRepublicanwashingtonstateeconomicdevelopment.vpweb.com
Mike the MoverRepublicantheoriginalmikethemover.com

For United States Representative
NamePartyWhy you should vote for
Don RiversDemocraticdonriversforcongress.com
GoodspaceguyEmploymentwealthcolonizespace.blogspot.com
Scott SutherlandGOPvote-wa.org
Andrew HughesDemocraticandrewhughesforcongress.com
Jim McDermottDemocraticmcdermottforcongress.com
Ron BemisRepublicanronbemisforcongress.org
Charles AllenDemocraticcharlesallen2012.com

For Washington State Governor
NamePartyWhy you should vote for
Rob HillDemocraticNo Website Found
Rob McKennaRepublicanrobmckenna.org
Jay InsleeDemocraticjayinslee.com
James WhiteIndependentwhiteforgovernor2012.com
Christian JoubertNo Party Preferenceholisticgovernor.com
Shahram HadianRepublicanhadian2012.com
L. Dale SorgenIndependentimagineliberty.us
Max SampsonRepublicanNo Website Found
Javier O. LopezRepublicanWebsite Down

For Washington State Lieutenant Governor
NamePartyWhy you should vote for
Glenn AndersonRepublicanglennanderson2012.org
Brad OwenDemocratbradowen2012.com
James Robert DealNo Party Preferencefluoride-class-action.com
Bill FinkbeinerRepublicanbillfinkbeiner.org
Dave T. Sumner, IVNeopopulistNo Website Found
Mark GreeneDemocracy Independentbrandnewelections.us

For Washington State Secretary of State
NamePartyWhy you should vote for
Jim KastamaDemocraticjimkastama.com
David J AndersonNo Party PreferenceNo Website Found
Sam WrightHuman Rightsthehumanrightsparty.org
Karen MurrayConstitutionmurray4sos.org
Kathleen DrewDemocratickathleendrew2012.com
Kim WymanRepublicankimwyman.com
Greg NickelsDemocraticgregnickels.com

For Washington State Treasurer
NamePartyWhy you should vote for
Jim McIntireDemocraticjimmcintire.com

For Washington State Auditor
NamePartyWhy you should vote for
Troy KelleyDemocratictroykelley.com
James WatkinsRepublicanwatkinsforauditor.com
Mark MilosciaDemocraticmarkmiloscia.com
Craig PridemoreDemocraticcraigpridemore.com

For Washington State Attorney General
NamePartyWhy you should vote for
Bob FergusonDemocraticelectbobferguson.com
Reagan DunnRepublicanreagandunn.com
Stephen PidgeonRepublicanstephenpidgeon4ag.com

For Washington State Commissioner of Public Lands
NamePartyWhy you should vote for
Stephen A SharonNo Party PreferenceNo Website Found
Peter J GoldmarkDemocraticpetergoldmark.com
Clint DidierRepublicanclintdidier.org

For Washington State Superintendent of Public Instruction
NameWhy you should vote for
James Bauckmanjamesbauckmanforspi.wordpress.com
Randy I Dornrandydorn2012.com
Don HanslerNo Website Found
John Patterson Blairjohnblairportal.wordpress.com
Ronald L (Ron) Higginswww.higgins-spi-2012.com

For Washington State Insurance Commissioner
NamePartyWhy you should vote for
John R AdamsRepublicaninfojohnadams.com
Mike KreidlerDemocraticmikekreidler.com
Scott ReillyRepublicanscott-reilly.org
Brian C BerendIndependentNo Website Found

Tuesday, July 17, 2012

Sage/DREAM Breast Cancer Challenge Launch

This morning's webinar marked the public launch of the Sage/DREAM Breast Cancer Challenge.  The goal is simple:  given a large data set consisting of clinical covariates, copy number data, expression data, and survival data, can you produce the best predictions of survival time for new patients?  The unstated goal is a little bit more subtle:  so far, there is no compelling evidence that I have seen that microscopic information such as gene expression or copy number data adds anything to predictive power in macroscopic behavior such as the survival time of the patient.  We hope that this competition will encourage people to build better models of disease collaboratively, and produce some of the first evidence that gene expression and copy number data can actually be useful in predicting patient prognosis.  As an example, I will produce an entry that you can feel free to cannibalize, and will write more about it as I evolve it into a better model of disease.  As a Sage employee, I am not eligible to win and so I hope you will modify and adapt this code and produce your own entry, superior to my own!

Before trying my code, you will want to check out the Breast Cancer Competition Getting Started Guide.

From an internal competition we ran that preceded this launch, we learned that the R RandomSurvivalForest package produced the best models of survival given the data, and that in fact the most critical part of the model was actually which data is chosen for inclusion and how it is pre-processed before being handed over to RandomSurvivalForest.

First, I will produce the core of my submission, the model class file.  I will take only the clinical covariates and totally ignore the copy number and expression data, though I will leave a space where they can be included later.  In fact, you will find that this transformation function is the most important part of the entire competition:  victory will hinge not on the best model, but instead on cleaning up the data in the best way possible.

View my Model Class File on github.

Next, I will train and submit the model.  This code is relatively straightforward, but if you are going to modify my model for your own purposes, you will want to make sure that you submit to the public leaderboard instead of the Sage leaderboard!

View my Submission File on github.

As you can see, my Random Survival Forest Clinical-Only model did pretty well!  On the Sage employees leaderboard, model "Sauerwine RSFModel test 3" got a respectable test score of 0.71.  In my next post, we'll see how to improve that further.  At the time of this post, it actually does better than any model on the Sage/DREAM Public Leaderboard, but since you're free to take my model and improve on it, that's not likely to be the case for long!

Good luck, and happy modeling!

Sunday, July 15, 2012

Keep it Simple!

I've updated my CV to reflect some success I've had lately in TopCoder Marathon Matches.  I competed in the Harvard Medical School #3 Competition, where I did dismally, and the NASA Tournament Lab "SynchronousControl" Weekend Competition, where I got a 4th place award.

I learned a lot from these contests, so I'll comment a bit on how they went and how I could have improved.

In the HMS #3 Competition, users were asked to examine simulated gene marker time series data to determine which genes were being most strongly selected for and against.  This boiled down to doing a regression on some data points from a Markov chain, and trying to determine what the parameters of the Markov chain were.  In principle, it was actually pretty straightforward.  Simply returning "parameter = (x1-1)/x0" actually did pretty well--better than my solution!  I had tried several different and more advanced regressions, as well as methods to try to account for uncertainty on the data points.  Ultimately, it turns out that all of this complex math did slightly worse than the obvious solution stated above.

The question itself had some problems, in my mind.  First, they assume that the reading at any given time point was correct and free of noise for the purposes of the next iteration of the Markov chain.  Second, the nature of the problem was such that you could not nominate a gene for being both a "most up-regulated" and "most down-regulated" gene.  Sometimes, the noise turned out to be so extreme that my best guess would have been to nominate a gene for both categories!  Finally, the simulated data was modified so that no gene would ever have a reading of zero, which is not realistic given the nature of the experiment.  So, I'm not entirely certain that the researchers who ran the contest are ultimately going to find that the solution is what they actually wanted in the first place.

What I should have done, in retrospect, is the following:  First, I should have immediately dissected and worked backwards from the data simulation program.  The problem description was not exactly representative of the way the data was being simulated, and understanding this earlier certainly would have affected my progress.  Simulating a lot of my own data for my own test harness also would have been instrumental.  Second, I found that very complex methods are actually not all that much better than very simple methods.  I should have taken all of the simple models I tried and compared them in parallel to simulated data and understood the domains in which each model worked the best, then attempted to identify the domain that each data set was in and used the best simple model per task.

So, I say that I did "dismally" because the extremely complex model I settled on was embarrassingly worse than the simplest possible model.

To be fair, the NTL competition that I got a 4th place award in didn't go a lot better.  The problem was to determine what moves to make to help a bunch of robots escape from a maze, under the condition that all of the robots must make the same set of moves!  The principal challenge was that the program to find these moves had to complete in under 2 seconds, which left time for just one A* or BFS in the largest possible map.  I spent way too much time working on a solution that I had to discard because it performed more searches than could be conducted in 2 seconds.

If I had realized this earlier, I would have focused on a heuristic to help robots cooperatively reach their goals instead of focusing on ways to greedily get each robot out individually in the best time.

The algorithm I ultimately used was this:  "Identify the closest robot to any goal.  Perform the minimum number of moves to get that robot out."  This crude method was sufficient to get a 4th place award in my room, $50.  What I would have done if I had to do it again would be this:  "Identify the closest robot to any goal.  Perform the move that minimizes the average distance between all robots and any goal, subject to the constraint that the closest robot must either move closer or stay the same distance away."

The overall lesson was this:  keep it simple.  I often find myself gravitating towards some complex custom algorithm which may either grossly overfit in the case of the HMS competition or may spend way too much processor time grinding away towards a solution when a crude heuristic could have approximated a better solution much more quickly.  When working on these open-ended problems, I need to fight the urge to write a complex algorithm, stay focused on the goal, and home in on the best solution using increasingly better heuristics.


Sunday, July 8, 2012

Vacation: Olympic National Rainforest

For her birthday, I flew my friend Lilli over to Seattle to explore the Olympic National Rainforest.  I'd seen pictures of the idyllic mosses and glaciers near the Hoh river, and I must say that it's something you have to see in person.  The forest is incredibly lush and green, but it's often hard to take a good picture because it seems like something is always in the way of your shot, and it was difficult to find wildlife because there were way too many hiding spots.  To make matters worse, I grossly underestimated how cold it got there at night, and overestimated how waterproof my tent would be.  I'll have to try to make the 18 mile hike up to the glacier meadows some other time!  The park as a whole is wonderfully maintained, and the rangers and park staff were pleasant and helpful--just make sure you come prepared for the moisture, cold nights, and permethrin-resistant mosquitoes.

A columbine flower, Aquilegia formosa


Scaphinotus angusticollis looking for a meal on a mossy trunk.
The elk were quite unafraid of people.
The trees frequently had hollow openings under them.  This effect occurs because the only sunlit real estate on the forest floor is where another tree has recently fallen!  This tree began its life growing on the trunk of a fallen predecessor, and now that progenitor has rotted away.