Hacking away with Linux, Ruby, Rails, and so on and so forth
13 Mar
After 10 weeks, I’ve now been able to watch all of the final presentations from the different groups. Considering that my group was the only “experienced programmer” team, I really didn’t know what to expect from the others. All in all, I’ve been incredibly impressed by the various projects they’ve managed to produce. Most importantly, it’s clear that the quality of the project ideas has nothing to do with programming ability. Implementation is another thing, but even groups of self-proclaimed “newbies” with only (prior to this class) basic HTML exposure came out with some very cool projects.
Something I hadn’t expected, but makes sense in retrospect, is the correlation between prior experience and willingness to learn new things. The class represented a huge cross section of prior-experience from student to student. Overall, those who knew the least starting out were the ones most willing to embrace new ideas and technologies. The eagerness with which the “beginners” jumped into their ideas was truly inspiring.
I spent a decent amount of time in and outside of class evangelizing Rails since it was so appropriate for a lot of the project ideas out there. This had different effects on different people. One group actually ran with it and built their entire project with it. A few people were probably bored to tears by my rambling. The majority of the class now knows me as the “Ruby on Rails guy”. I’m glad that I was able to expose more people to the framework, but I don’t think I was effective in conveying the core concept that I work by:
If you’re going to tackle a problem, use the best tools available.
Rails is a great tool for a wide variety of web apps. It is not the be-all end-all solution to life’s endless problems. I think I came across more as “Rails is the answer to everything” instead of what I was shooting for “Use new tools, not just what you already know.” A large part of this problem stems from our own group using Rails for the backend; something I was actually trying to avoid.
The aim of the class is for each person to learn new technologies and develop something with them. Our project, due to the nature of its complexity, was going to require a lot of coding. Once it became clear that we were going to need our own backend DB, I wanted to avoid using PHP. Not because it wouldn’t work just fine. There’s nothing wrong with it, especially for a project this small. The problem, as I saw it, was that 4 out of 5 people in our group had a medium to high level of experience with it. That doesn’t allow for a lot of “new” learning.
My motivation for using Rails in our group was to introduce already-skilled programmers to something new, even if it meant limiting my own exposure to “new” things. For my own learning, I made sure to take time outside of our group meetings to teach myself other pieces that I haven’t seen or used before. The most visually interesting result of that is the drag-n-drop ordering of Categories. Working out the SQL queries needed to find stores that are open at a given time definitely stretched my boundaries. Obviously not as major as learning a whole new language, but it was a great way to push myself beyond what I already knew coming in.
Were I to do it all over again, I would have spent more time trying to preach the core philosophy of “use the best tool” instead of keeping the focus on Rails. For example, I only realized last week that we, it appears, are the only group using any kind of version control. Giving a talk on basic SVN usage, I think, would have been valuable to the class as a whole.
I had a lot of fun working on Open Past Midnight for this class. After all, I’m a programmer. But the neatest part of this class was getting to see what other people came up with, especially when they had to work with limited experience. That constraint can be incredibly discouraging, but they pulled through beautifully.
3 Mar
It’s been too long since the last update, especially since we’ve overcome a lot of hurdles in building this system. I’ll talk about each in its own right.
The primary quandary we faced was in how to represent a store’s hours of operation in the database. The obvious implementation is to have separate columns for each day’s open and close times. This gives us 14 columns added to the ’store’ table, named ‘monday_open’, ‘monday_close’, ‘tuesday_open’… and so on. Since we have a different column for each day, it would make sense to have the column be of type ‘time‘ right? No!
Especially because we’re dealing with locations that are open late, we need to deal with cases like “On Monday, we’re open from 10am - 2am”. If the columns only handle time, then we’ll have a store with monday_open = 10:00 and monday_close = 2:00. That means we need to do some serious condition checking to see if its open. Better yet, how do we query for all stores that are open at a specific time? Not very efficient.
So how did we deal with this? We want the columns to be able to know not just the time of day, but also the day of the week. For the example above, we want the columns to read more like monday_open = Mon 10:00 and monday_close = Tue 2:00. This would make queries a whole lot easier, as we can resort to a simple open < right_now < close test, without having to create extra logic. But SQL databases don’t have a type to represent time + day of week. The options are time (hours, minutes, seconds only) or fullblown datetime (year, month, day, hour, etc). Since we’ve concluded that time is not enough on its own, we’re forced into using a full datetime field.
But wait! That means monday_close will look more like March 4, 2008 2:00 -0800. How are you going to deal with the comparisons when it’s now June? The condition will always fail, thinking the store closed months ago! And that’s where our solution comes in. The solution is, to some degree, in the question. Our comparisons will involve the entire datetime, but we only care about day-of-week and time. Therefore, we just need to pick arbitrary values for the rest of the datetime fields and have the code enforce their usage. We ended up using the week of Jan 1, 2007 for our project, primarily because Jan 1 is a Monday, making it very easy to translate a number to day-of-week. Throw in a few helper functions to make the date translation seamless to the user, and we’re set!
As I said in my previous post about web hosting companies, DreamHost is a great cheap shared-hosting solution, but its Rails support leaves a bit to be desired. I was pleased to see that they’ve cleaned up their post about web hosting companies, DreamHost is a great cheap shared-hosting solution, but its Rails support leaves a bit to be desired. I was pleased to see that they’ve cleaned up their documentation, and getting a Rails site running requires a lot fewer hacks now. But the speed issue is still a huge one. Big enough for me to still say that for a commercial Rails app, I would not use them. For a class project, they’re perfect.
We’ve transitioned the project to Rails completely, which has made, as usual, a lot of grunt work disappear into thin air. It’s working well for what we need from it, albeit a very simple site. The real issue with Rails is that, being so new, still very few people know about it, much less have used it before. Whenever introducing a group of people to a new technology, you’re going to have mixed results, and our team is no exception. Some have taken to it excitedly, some with reserve, and some are just not very interested. All this is fine, especially since there’s plenty of work to do outside of Rails. It’s interesting to have such a cross-section of reactions to it all working together.
Since our project employs Google Maps to display store locations, we need to interface with their API. More importantly, we need to load latitude and longitude coordinates of the stores into our own database. Since we all quickly agreed that we didn’t want to do that by hand, we worked out an alternate solution.
The interface to add a new store to the database employs Google Local Search to find stores that are already in Google’s system, and therefore already have all of the information we need (except hours). From the search results we get back from Google, we can click a button that then pushes the data up to our own server. Then we go in and set the hours of operation and assign a category. It took a lot of tweaking to get it working right, but after a few hours of Rodolphe working on the client-side Javascript and me working on the server-side receiving end, we got it ironed out.
19 Feb
For those of you that have not heard, my grandfather is very ill and in the hospital. I’ve been back and forth to LA several times this last week to try and help out the family as much as I can. Even better, I picked up a really nasty cold somewhere along the way. All in all, I’ve missed a bunch of classes, a few assignments, and a midterm. I’m playing catchup, first and foremost on Open Past Midnight. It’s one thing to get behind on my own work, but to set back my whole group is not acceptable.
That said, we’re transitioning the Open Past Midnight project to Rails since we’ll need to be dealing with our own database. I’ve already got Evan set up with the dev environment. Then again, he’s running Leopard, so it took all of 5 minutes. The other teammates all use Winblows, so it might take some extra time. Either way, I am eternally grateful that Rails 2.0 now uses SQLite as the default DB. No longer will I need to take time to explain MySQL privilege tables to my classmates when all they want to do is some basic development.
More to come in later posts, but we’re still going to try and deploy this project on Dreamhost. Since it’s hardly a commercial endeavor, I’m not as worried about DH’s poor Rails support. It will be interesting though to see how far they have (or have not) come.
3 Feb
Due to some slight difficulties in acquiring the needed domain name, our COGS group has changed its name to Open Past Midnight. We’ve been working on some paper-prototypes for the site layout. Vivien found a very cool possible technology for generating heatmaps called gheat. Given the implementation, I don’t think it will be directly usable for our project, but it does lend some great ideas. There might be parts that we can yank out and use. We’ll see.
I’ve also been working a lot on the server-side systems we need to make this project run smoothly, both in development and production. Just as with Rails, Capistrano is my friend. I’ve almost got it working for this project, even though we’re likely to do any back-end code in PHP.
Also, it turns out that Google will pay you to submit verified listings of local businesses. They call it their Business Referral Representative program. If we’re lucky, we can get into the program and get paid for doing all the data gathering we would anyway. More to come.
27 Jan
So our COGS group has decided to do a Google-maps based site that allows the visitor to find nearby restaurants (maybe stores, etc) that are open at different times of the day. We’ve set up some framework stuff (SVN, domain, etc) and need to start getting things together.
By Thursday (1/31/08) we will have mapped out the data for the following locations. This means we need to go to those areas and grab all of the data about the various stores (name, phone, address, hours) and submit it to Google’s database (or verify that they already have it.)
Our in-progress feature list:
Must-Have Feature Set
Would-Be-Nice Feature Set
22 Jan
The first assignment for COGS 121 was due last week. I worked up a simple site on Practical Chinese for Nerds. The content leaves a lot to be desired, but that was essentially filler for me. I was focusing on two things while I built this site.
The thing I’m really looking forward to playing around with is OpenLaszlo. It looks very powerful, and I’ve got a few ideas for projects where I could implement it. The most interesting would be to use Laszlo’s DHTML output to create an interface to a Rails app. Muhahaha.
17 Jan
Finally! After very popular demand, I’ve managed to post starcupsdrv_2.3.0-0ubuntu1_i386.deb just for you! Since WordPress is not nice about uploading .deb files, I’ve put it on a separate sub domain with a direct download link. No special scripts or sign-in required.
If you download and use the deb file, I’d appreciate if you could leave a quick comment on this post to say hi. If the deb is really that popular, I’ll work on expanding support for it.
Once you’ve downloaded the deb, you can install it with the following command:
dpkg -i starcupsdrv_2.3.0-0ubuntu1_i386.deb
9 Jan
So in my 5th and final year at UCSD, I’m finally getting to take a class that I’ve had my eye on for years: Human Computer Interaction Programming. Part of the class assignments are to maintain a blog of what we do in (or related to) class. Since I’m lazy and don’t want to set up a whole new one, here we go!
This should also motivate me to continue my regular postings. For those readers that have been patiently waiting for the printer driver I talked about here, it will be posted this week.
18 Sep
One of my philosophies in life is to set personal goals that seem almost impossible to reach. Then reach them. Yes, it sounds ridiculously cliche, but it moves me forward. One such goal of mine was to break 8 bricks. While I didn’t quite nail it last November, I got close.
This year, I plan to try my hand (literally) at 9 instead.
17 Sep
Following up on my post about installing a receipt printer on Ubuntu, I mentioned that I would post my experiences setting up a Ubuntu repository. Most of the documentation out there only covers setting up a repo on your own local machine, for your own local use only. On top of that, the docs vary between how much they tell you about the specifics of a Ubuntu vs Debian setup. Since I am likely not the only one trying to make this kind of thing, I’m posting a step-by-step guide on how to do it yourself.
We want to setup a publicly accessible Ubuntu repository so that we can distribute our own .deb packages to other users. Since we’re on a budget, we’re going to host this repo on our DreamHost account. Above all else, it needs to be easy to maintain, since we want this to make life easier, not harder.
We’re going to create a local repository on our development machine using dput and mini-dinstall. Then we’ll sync it to the remote web server using rsync and ssh.
As always, this guide assumes that you are familiar with Linux and are comfortable doing things on the command line. If not, turn back now, before it is too late! This is not a guide for beginners.
We’re also going to assume that you’ve got a hosting package with DreamHost, and a domain name to which you can add sub-domains.
Lastly, we’ll assume that your development machine is running some form of Ubuntu Linux. I used Feisty (7.04) when writing this article. Adjust the steps as needed for your own setup.
To keep your website clean, it’s a good idea to set up a sub-domain of your primary site on which to host your repo. Using my site www.drelmo.net as our example, we set up packages.drelmo.net as our domain. On DreamHost, you want to set this up as a Fully Hosted Domain. If you know what you’re doing, you can customize the options on the setup screen, but the defaults are all fine.
To make life easier down the road, we’ll need a password-less SSH login to the site. I won’t go into the details of how to do this, as it has been documented far better in countless places. The DreamHost wiki, for example. I’ll assume from here on out that you can simply type “ssh user@server.dreamhost.com” and get dropped into a bash shell.
Again, if you don’t understand that last sentence, stop now, before you hurt yourself.
To keep things clean, I like to segment things a bit. This is an optional step, but one that I suggest doing unless you have a really good reason not to. Log into your server real quick and in the webroot, make a folder called ‘archive’. In the case of our example, this would go something like this:
ssh user@server.dreamhost.com cd packages.drelmo.net/ mkdir archive exit
I know, I know, we don’t care about having a local repo, we want to share it with the world! Sit tight, junior. This is part of the process. On shared hosting, you likely won’t have access to the tools needed to easily maintain a repository, and for good reason. So we’re going to set it up (and yes, keep it there) on our development machine first.
Rather than rewrite it, I’ll point you to the excellent guide that I followed called LocalAptGetRepository, over at the Ubuntu Community site. Follow that all the way down to (but not including) the section called “Changing your systems apt-get sources.list“.
All done? Great! Let’s move forward.
At this point, you should have a folder somewhere on your dev machine that looks something like this:
/home/marc/archive /home/marc/archive/feisty /home/marc/archive/gusty /home/marc/archive/mini-dinstall
And in those folders should basically be everything you need for your repo, hopefully with one or more packages already imported. If not, I suggest getting to that point before you move on.
We can simply type in the rsync command to get all our files up there, but we want this to be easy later on, remember? So let’s make a simple bash script that can do it all for us, so we don’t need to constantly retype the whole thing. I recommend calling it something simple like “sync-repo“. Stick the following into it, making sure to adjust for your setup.
A few notes about the script: Wordpress messes up the display of the rsync command, so there are two things to pay attention to.
- The three lines, starting with the rsync one, should all be on one line.
- The long dashes in front of “stats”, “progress”, and “exclude” should actually be two normal dashes.
#!/bin/bash rsync -rlptvzh –stats –progress –exclude=mini-dinstall -e “ssh -l user” /home/marc/archive/ user@server.dreamhost.com:/home/user/packages.drelmo.net/archive/
Now save the script, chmod +x it, and run it! If all went well, you should be able to point your browser at the equivalent of http://packages.drelmo.net/archive and see the same folder tree as the one on your machine. You’ll notice that we excluded the mini-dinstall folder, because that should not be made public.
The final step is to add your brand new repository to your /etc/apt/sources.list. Open up the list in your favorite editor:
sudo vim /etc/apt/sources.list
Then add in the lines for your repo:
deb http://packages.drelmo.net/archive feisty/ deb-src http://packages.drelmo.net/archive feisty/
Update your cache and you’re done!
sudo apt-get update
If you’ve followed along, you should now have your very own Ubuntu repository. To add new packages to it, just use dput on your local machine, then run your sync-repo script to update the online version. It’s that easy!
Recent Comments