Feb. 12, 2018, 12:36 p.m.
A few months ago I went to a university to interview for a job working on their website. Up until that day I had been programming in PHP with Laravel and Symfony. These MVC frameworks are object-oriented, and they make PHP seem like a "real" programming language. I love them for that. Everything is nicely organized and segmented and you can do almost all the programming in an object oriented fashion.
The university was using a CMS and they asked me to take a little test by looking at some of the code. The code was written in non-object-oriented PHP, which means that everything was done in the page. If you need data from the database you do the query right there in the page, do whatever manipulation you need, then loop through the results with everything embedded right in the HTML with <?php tags.
I was shocked and dismayed looking at the code. It was about as elegant as the BASIC code I was writing when I was 10 years old, but with HTML mixed in for good measure. It was horrifying to realize that this was the language I was programming in, despite the fact that OOP frameworks make it into something that resembles nicely written code.
The other thing that happened that day was they asked me what my ideal job was. I said I would really like to be involved in some sort of research, but there was no need for PHP in that field. When I got home I thought about it for a few hours, then decided that if that was what I wanted to do I should learn whatever I needed to learn to do it. And that is what I did, and that is why I stopped programming in PHP.
Don't get me wrong - I'm not saying PHP is a horrible language, it can be used very well. But the majority of employers here in Switzerland looking for PHP programmers are not looking for people who do it very well, they are looking for people who have done a three month bootcamp and will work for very low pay. For me, it just wasn't worth the effort to do something I don't really even like that much.
Jan. 26, 2018, 10:36 a.m.
Python code to make sure two data frames have the same columns in the same order. I used this to make sure that two dataframes had the same dummy columns after using pd.get_dummies:
missing_cols = set( X1.columns ) - set( X2.columns ) for c in missing_cols: X2[c] = 0 X2 = X2[X1.columns]
Jan. 23, 2018, 11:25 a.m.
My favorite languages, in order:
Jan. 18, 2018, 11:16 a.m.
In 1998, when I graduated from university the internet was in its infancy and the dot com bubble was just getting ramped up. At the time I thought the internet would change the world by making information easily accessible and available, and I was excited to be a part of this new thing that would be revolutionary.
That started to change a few years ago. The focus of internet companies had shifted from providing useful services to collecting as much data as possible on the users to be used to better target advertisements. Rather than providing useful and informative content and services, the emphasis was on keeping users online for as much time as possible. While the negative effects of this business model on the users and society are becoming more and more noticeable, notably in the recent US election, the tech companies continue to ignore them. This is no longer the industry that I signed up to work for, and I no longer want to be a part of it.Anyone who is familiar with the work of Kahneman and Tversky knows that the human brain is very poor at processing and analyzing data. Most of our decisions are made using heuristics, or rules of thumb, that allow us to make quick and easy judgements. These result in cognitive biases, which are ways in which our brains distort reality for the purpose of making decisions. One of the most famous cognitive biases is the "confirmation bias" - which is how people interpret new information in a way such as to support their existing beliefs. Kahneman and Tversky conducted experiments on people ranging from college undergraduates to statistics professors, and everyone was subject to these biases - even PhD level statisticians who should know better. This is why data science is so important.
Our brains are not designed to gather and analyze large amounts of data and we are incredibly bad at doing so. We tend to draw conclusions from small, isolated, but memorable bits of information rather than looking at the overall big picture. One example is how Americans are all very worried about terrorism even though on average only six Americans die per year from foreign terrorism. The media likes to report these stories because they are sensational and memorable, but doing so greatly exaggerates the real risks. There are also numerous medications which are commonly prescribed despite having minimal positive effects, or having no benefits at all.
Data science is a way to draw knowledge from actual observation of the world, rather than just whatever thoughts happen to be strung together in our heads, or whatever sound bites relating to a given subject most easily come to mind. I can come up with whatever theories and ideas I want, but unless they actual reflect on the real world it's all meaningless. This is the basis of scientific inquiry, and this is why I am getting out of web development and into data science.