sqlite> select count(*) from posts where external_id = 199252;
0
sqlite> select count(*) from posts where external_id = '199252';
1
sqlite> select count(*) from posts where external_id = 199793;
1
sqlite> select count(*) from posts where external_id = '199793';
0

One day I suppose I will find it useful to store different data types in the same column, and without implicit conversion. I look forward to that day. Right now, I will have to spend a few hours cleaning up the mess that this caused.

Apparently I should have let go of Fedora 6 some time ago. I installed this particular server in December 2006 when FC6 was the cool new girl on the block, and it has been running smoothly since then. But today when I tried to (yum) install a package, there were no servers to be found:

# yum search libxslt-devel
Loading "installonlyn" plugin
Setting up repositories
http://archive.kernel.org/fedora-archive/fedora/linux/core/6/i386/os/repodata/repomd.xml:
  [Errno 14] HTTP Error 404: Server: nginx/0.5.0
...
Trying other mirror.
core                      100% |=========================| 1.1 kB    00:00
Error: Cannot find a valid baseurl for repo: extras

Oops. I looked in /etc/yum.repos.d/fedora-extras.repo which is provides the base URL:

[extras]
name=Fedora Extras $releasever - $basearch
#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/extras/$releasever/$basearch/
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=extras-$releasever&arch=$basearch
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-extras
gpgcheck=1

In earlier experience, the problem has been that the DNS service for the repo that mirrorlist recommends is down, and this is what some googling suggested too. So I went to mirrors.fedoraproject.org to see what it said. The answer surprised me:

# repo = extras-6 arch = i386 error: invalid repo or arch
# following repositories are available:
# repo=core-2, arch=i386
# repo=core-2, arch=x86_64
# ...
# repo=core-6, arch=x86_64
# ...

So core-6 is in the list but extras-6 is not. I tried some different combinations thinking I had got it wrong, but to no avail. Instead, on a whim, I checked out the site’s root where I could navigate to a human-readable list of mirrors and I started checking them out one by one. Uni-Bayreuth seemed to have what I wanted, so I edited fedora-extras.repo like this:

[extras]
name=Fedora Extras $releasever - $basearch
#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/extras/$releasever/$basearch/
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=extras-$releasever&arch=$basearch
baseurl=http://ftp.uni-bayreuth.de/linux/fedora/linux/extras/$releasever/$basearch/
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-extras
gpgcheck=1

and ran yum search libxslt-devel again. This time it complained about updates instead (Error: Cannot find a valid baseurl for repo: updates). Unfortunately Bayreuth didn’t have the updates catalog which sent me searching in a few other places until I finally decided to try to make do without it by setting enabled=0 in fedora-updates.repo:

[updates]
name=Fedora Core $releasever - $basearch - Updates
#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/updates/$releasever/$basearch/
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-fc$releasever&arch=$basearch
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora

It worked, and I was able to install the packages I wanted, but I really need to get myself a newer OS. I’ll combine that with a move to another host; FS-data have let me down once too many, and I’ve never fancied upgrading the OS on a production server anyway.

1. checkout the previous revision,
2. restart the server (because plugins, gems and even Rails can change between versions),
3. try to log in,
4. see if works and stop if it does,
5. repeat from 1.

With git, I can ask git itself to perform a binary search among all the revisions between now and three weeks ago. It would go like this:

1. git bisect start HEAD 57c3aaa,
2. restart the server,
3. try to log in,
4. see it fail or succeed,
5. say git bisect bad or git bisect good to make Git checkout the next node in the binary search tree
6. repeat from 1 until Git says that it found the culprit.

It doesn’t look a lot better, but it is if you have 300 commits. Of course there is nothing to stop you from doing a manual binary search with Subversion, or even to step back one day at a time to narrow it down, and we didn’t actually have 300 commits. But where Git really shines is in the fact that if I can write a program that can tell if a version is good or bad, all I have to do is

1. git bisect start HEAD 57c3aaa,
2. git run ../my-test-probe.rb

and sit back and wait. Provided of course that the program also knew how to start and stop the server. Well, here is that program, for your debugging pleasure, and mine:

$ ps -fe | grep rails
UID        PID  PPID  C STIME TTY          TIME CMD
wcj      27292  3558 27 13:41 ?        00:00:02 RAILS: /home/w/wcj/rails/sis/public/dispatch.fcgi
wcj      27365  3558 40 13:41 ?        00:00:02 RAILS: /home/w/wcj/rails/sis/public/dispatch.fcgi
...

With more than two processes I refuse to kill by PID – FCGI is just too quick to spawn new processes, and I’m not sure if they will run the right version of my app either. So it’s killall. But what to killall?

$ killall -9 'RAILS: /home/w/wcj/rails/sis/public/dispatch.fcgi'

does not work. So it’s ps to the rescue:

$ ps -eo pid,comm:30
  PID COMMAND
27365 rails_dispatche
27292 rails_dispatche

rails_dispatche, really? No final ‘r’? Whatever:

killall -9 rails_dispatche

It works and I’m happy again.

And then I read today’s Bloom County strip (re-runs from the ’80s) where Opus arrives at the barber’s for his “biannual haircut” and I decided to take action.  I can deal with not knowing Malcolm Gladwell’s party schedule, but I really want to know if my favourite penguin gets a trim twice a year or every two years.  (I have tried both intervals, and each has its charm.)

I started with “biweekly” and here’s what the dictionary says:

biweekly |bʌɪˈwiːkli|
adjective & adverb
appearing or taking place every two weeks or twice a week : [as adj. ] a biweekly bulletin | [as adv. ] she followed her doctor’s instructions to undergo health checks biweekly.

Wow.  I never finished my course in Language Philosophy in 1992, but I remember the Morning Star Paradox quite well.  It goes something like this:

• Mary belives that the Morning Star is Lucifer, and she believes that the Evening Star is the goddess Venus.  Her friend Tycho is aware of her beliefs, but…
• Tycho knows that the Morning Star and the Evening Star refer to the same physical object, so…
• Tycho thinks that Mary believes that Lucifer and Venus are one and the same.

I remember that this was a useful tool for understanding the difference between a reference and a referent or somesuch, but most of all I thought it was beautiful.  Mary has romantic beliefs about the existance of deities and their position in the sky, and while scientists normally shrug at romantic beliefs, they (we) all have to stick our noses into this one.  Dear Mary, you can believe that the Morning Star is Lucifer, and you can believe that the Evening Star is Venus, but you can’t believe both.  If you can’t see the beauty in this I probably can’t help you.

Anyway, here are today’s more or less contradictory words:

biweekly: (adj,adv) every two weeks or twice a week; (noun) a periodical that appears every two weeks or twice a week

bimonthly: (adj,adv) occurring or produced twice a month or every two months; (noun) a periodical produced twice a month or every two months.  (With the added note that “[i]n the publishing world, the meaning of bimonthly is more fixed and is invariably used to mean ‘every two months.’”)

biennal: (adj) taking place every other year, (noun) 1 a plant that takes two years to grow from seed to fruition and die. 2 an event celebrated or taking place every two years.

biannual: (adj) occurring twice a year

There are a few other things that come to mind when I see these words together.  One is that ‘biannual’ is never a noun.  Another is that the lexicographers chose to say “every two weeks or twice a week” but “twice a month or every two months”.  Order is not always significant in a dictionary, but it’s hard to fathom that not at least one of the two entries has a bias for ‘twice’ or ‘every two’.  But it’s of course impossible to know which one it is.

And thirdly, the note for ‘bimonthly’ as a noun makes me wonder if the notes are written by a different group of people from the ones who write the actual definitions, or if ‘publishing world’ is the key word here.  You and I might call a periodical that appears every two weeks bimonthly since it appears twice a month, but in the publishing world, it will be a biweekly instead.

Wow.

I parked my bike on Magasinsgatan last night in order to watch Rabalder perform Requiem. (The concert was a treat in a way that would have made Mozart green with envy.  He would have died to have electric guitars in his setting!)  After the concert, I left the bike there and chose to walk to the next event (this being Kulturnatta) together with friends.  I live three blocks away and planned to pick up the bike on my way home.  Of course, I pride myself by not living by plans, and when picking up the bike on the way home turned out to be a six block detour, I decided to leave it for today.

I’ll have to walk more than six blocks to the bike shop to have the three kicked-off spokes replaced, though.  Hopefully it will be cheaper than having the front wheel replaced, like last time, or replacing the night lights the time before that.

There is a patch coming out in a few weeks (or it may just be included in Flash 10 which I suppose will be a mandatory upgrade).  Fine again.  But how will I remember to go back and restore Flash’s access to my microphone and camera?  Well, hopefully I will see this blog entry the next time I write something.  Note to self.

Source: Macworld.

#! /usr/bin/env ruby
require 'rubygems'
require 'hpricot'
require 'open-uri'

doc = Hpricot(open("http://example.com/the_file.xml"))
(doc / :person).each { |person| ... }

Couldn’t be simpler. This time, there is a snag: the file is sensitive, so the connection is encrypted using HTTPS. For this article, let’s say we’re talking about Cert’s list of new vulnerabilities, which can be found at https://www.cert.org/blogs/vuls/rss.xml. open-uri supports HTTPS, so it shouldn’t be a problem, but it is:

doc = Hpricot(open("https://www.cert.org/blogs/vuls/rss.xml"))
# =>
/usr/lib/ruby/1.8/net/http.rb:590:in `connect': certificate verify failed (OpenSSL::SSL::SSLError)

OpenSSL, which open-uri uses behind the scenes, fails to verify Cert’s certificate and halts execution.

Solution 1: skip verification

Let’s assume that I don’t care much about the verification; all I want is the data, and it just so happens that it is only available through HTTPS. open-uri doesn’t let me turn off verification so I have to dig deeper.

open-uri is just a clever wrapper around Ruby’s comprehensive, but insufficiently documented, networking library that handles a variety of protocols, including HTTPS. To fetch a web page over a secure connection, you can use something like this sample client (from net/https.rb):

#! /usr/bin/env ruby
require 'net/https'
require 'uri'

uri = URI.parse(ARGV[0] || 'https://localhost/')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true if uri.scheme == "https"  # enable SSL/TLS
http.start {
  http.request_get(uri.path) {|res|
    print res.body
  }
}

There are three things to note in the sample client:

1. You should require net/https, not net/http.
2. You create the client with Net::HTTP.new, not Net::HTTPS.new. (There is no HTTPS class despite the fact that you require 'net/https'.)
3. You need to set use_ssl = true explicitly. The URI library is clever enough to set its port attribute to 443 when it parses a URI that starts with https, but Net::HTTP isn’t quite as clever.

If you put the above code in webclient.rb and run it, you’ll see this:

$ ruby webclient.rb https://www.cert.org/blogs/vuls/rss.xml
warning: peer certificate won't be verified in this SSL session
<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
    <channel>
        <title>Vulnerability Analysis Blog</title>
[...]

Yes, it will fetch and print the RSS XML, but it will also warn you that it doesn’t verify the host’s certificate. Let’s turn off the warning by telling Net::HTTP that we don’t expect it to perform any verification:

uri = URI.parse(ARGV[0] || 'https://localhost/')
http = Net::HTTP.new(uri.host, uri.port)
if uri.scheme == "https"  # enable SSL/TLS
  http.use_ssl = true
  http.verify_mode = OpenSSL::SSL::VERIFY_NONE
end
http.start { ... }

Run this, and you get the same result without the warning.

Solution 2: add verification

Solution 1 is not enough for my current needs. I want encryption, but I also want to know that I’m talking to the right server. To turn on verification, I change VERIFY_NONE to VERIFY_PEER and run again. Now I’m back on square one with OpenSSL::SSL::SSLError: certificate verify failed. Uh-huh. So what’s wrong with that one? It works in my browser without problems.

I’m not going to go into how HTTPS and certificate validation works. Suffice it so say that my browser is more trusting than OpenSSL. And it’s not blind trust either; the browser knows more Certificate Authorities. So how do I add them to Ruby and OpenSSL? I looked around and found a solution to a similar problem, Connecting to POP3 servers over SSL with Ruby. Adapting that to my HTTPS problem, it becomes a two-step solution:

1. Download the CA Root Certificates bundle from haxx.se, the creators of curl. Store the file in the same directory as webclient.rb and make sure that it’s called cacert.pem. (But please see the discussion below on Too much trust.)
2. Make webclient.rb use this file instead of whatever is bundled with OpenSSL.

Now we can tell Net::HTTP to use this CA file:

uri = URI.parse(ARGV[0] || 'https://localhost/')
http = Net::HTTP.new(uri.host, uri.port)
if uri.scheme == "https"  # enable SSL/TLS
  http.use_ssl = true
  http.verify_mode = OpenSSL::SSL::VERIFY_PEER
  http.ca_file = File.join(File.dirname(__FILE__), "cacert.pem")
end
http.start { ... }

Look, it works! It gives the expected output, and it is verifying… something. But what? Time to look under the hood again. It turns out that with these settings, OpenSSL checks that the server certificate is signed by a known CA and has not expired, which is good, but not everything I’m looking for. I also want it to check that the certificate belongs to the server that I’m talking to. To see an example, go to https://google.com/. In Firefox 3, you should get an iconic policeman telling you it’s a Page Load Error. The certificate belongs to www.google.com, not google.com. But our script is not quite as discerning:

$ ruby webclient.rb https://google.com/
hostname was not match with the server certificate
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.com">here</A>.
</BODY></HTML>

Note the warning on the first line of output. Apparently Net::HTTP checks to see if the certificate belongs to the host, but it’s not a fatal error. To change this, we need to enable the “post-connection check”. So here is the final version of the script:

#! /usr/bin/env ruby
require 'net/https'
require 'uri'

uri = URI.parse(ARGV[0] || 'https://localhost/')
http = Net::HTTP.new(uri.host, uri.port)
if uri.scheme == "https"  # enable SSL/TLS
  http.use_ssl = true
  http.enable_post_connection_check = true
  http.verify_mode = OpenSSL::SSL::VERIFY_PEER
  http.ca_file = File.join(File.dirname(__FILE__), "cacert.pem")
end
http.start {
  http.request_get(uri.path) {|res|
    print res.body
  }
}

Now it will fail for https://google.com/ but succeed for https://www.google.com/. Done!

Too much trust

OK, I should admit that downloading a file from someplace called haxx.se doesn’t seem like the best way to raise security. If you really want to know who you will be trusting, you should download Root Certificates from each of the CA’s that you trust. That’s way too much work for the application I’m working on right now, but it might be a requirement for you. If you don’t want to go mad, though, try doing it the same way the haxx people did. They wrote a little tool to extract the Root Certificates from the source files of Mozilla, and they even have a tool for extracting it from your binary installation. Check out their documentation for a full description and links to the tools (source code).

There has been some commotion recently regarding XHTML and HTML, where standardistas have reiterated their arguments about XHTML: it’s a nice standard, but since is isn’t actually treated as XHTML by browsers, it’s better to use HTML.

The “standard” text on this (at least for me) is http://www.hixie.ch/advocacy/xhtml from which I’d like to quote the executive summary:

If you use XHTML, you should deliver it with the application/xhtml+xml MIME type. If you do not do so, you should use HTML4 instead of XHTML. The alternative, using XHTML but delivering it as text/html, causes numerous problems that are outlined below. Unfortunately, IE6 does not support application/xhtml+xml (in fact, it does not support XHTML at all).

I have come back to that text on several occasions, but it never quite convinced me. I didn’t want to lose the well-formedness of XHTML. Then I ran across http://www.webdevout.net/articles/beware-of-xhtml which made me change my mind. The article uses numerous examples to show that XHTML is not just a syntactically nicer version of HTML, but it actually has slightly differing rules for how to apply CSS. So basically, in all projects that I have used XHTML, I have been relying on bugs in present browsers. Ouch.

Time to change—back

They promised us that XHTML was the future, so I haven’t worried too much about HTML lately. Are there any downsides to using HTML instead? Well, not per se. But the tools and platforms I use are slightly XHTML-centric.

1. All the standard Rails helpers (form helpers, JS helpers, etc.) produce xhtml. The simple solution is to drop this in a file in lib/:

   module ActionView::Helpers::TagHelper
     def tag_with_html_syntax(name, options = nil, open = true, escape = true)
       tag_without_html_syntax(name, options, open, escape)
     end
     alias_method_chain :tag, :html_syntax
   end
   

2. TextMate snippets produce XHTML. Most of them can be controlled by an environment variable, but there are exceptions, like C-ret which produces produces an XHTML self-closed br element.

3. External tools, data sources and parsers are usually xml-centric.
4. WordPress, which I have used for a few client projects lately, spits out XHTML. I’ll see if I can create a patch for that and have it accepted.

Oh well. XHTML, I’ll miss you. HTML, can we be friends again?

def quote (str)
  str.gsub('\\','\\\\').gsub('\'','\\\'')
end

I read this as “replace all backslashes with double backslashes, and then replace all single quotes with a backslash and a single quote”. I added a simple test for it (yay TestUnit!):

def setup
  @m = Migrate.new
end

def test_quote
  assert_equal("I\\'m home", @m.quote("I'm home"))
end

But imagine my surprise when I got

  1) Failure:
test_quote(TestMigrate) [migration/test_migrate.rb:29]:
< "I\\'m home"> expected but was
< "Im homem home">.

Ooookay. What’s wrong here? Have I misunderstood the rules for escaping the escape sequence? It’s supposed to be easier with single quotes, but maybe I got it wrong. So I tried with double quotes:

def quote (str)
  str.gsub("\\","\\\\").gsub("'","\\'")
end

Surely this would work? Nope, it gives the exact same error. Time to look up gsub in the manual:

str.gsub(pattern, replacement) => new_str
str.gsub(pattern) {|match| block } => new_str

[…] If a string is used as the replacement, special variables from the match (such as $& and $1) cannot be substituted into it, as substitution into the string occurs before the pattern match starts. However, the sequences \1, \2, and so on [my emphasis] may be used to interpolate successive groups in the match.

“And so on”? Oh, so obviously \' (escaped \\' in the string literal) is the replacement string equivalent of $', which means everything afther the match (as all regexp hackers know). So I need to escape the backslash for regexp engine too:

def quote (str)
  str.gsub("\\","\\\\").gsub("'","\\\\'")
end

OK, the tests pass. But the code looks wrong. Four backslashes can’t work for both cases, can they? Let’s add a test case:

def test_quote
  assert_equal("I\\'m home", @m.quote("I'm home"))
  assert_equal("S\\\\N", @m.quote("S\\N"))
end

Nope, that fails. So we need this:

def quote (str)
  str.gsub("\\","\\\\\\\\").gsub("'","\\\\'")
end

Eight backslashes. Yes, the test passes, but is it worth it? Is it understandable? I don’t want comments to explain my code. Comments are good to provide a raison d’être for something, but not to explain its looks. Let’s switch to the other form of gsub:

def quote (str)
  str.gsub(/\\|'/) { |c| "\\#{c}" }
end

“If you see a backslash or a single quote, replace it with a backslash and whatever you saw.” That’s what I wanted to say anyway.

Good. But I wrote this in Markdown, so now I have to generate the HTML and the go through it and make sure that I restore whatever backslashes Markdown ate. (It turns out it didn’t eat any. TextMate has a Markdown Preview function that ate a lot of backslashes, but when I said “Convert to HTML” it didn’t eat any at all. Go figure.)

But that’s hardly a note to self.  Looks like I’m turning this into a blog.

sudo env ARCHFLAGS='-arch i386' gem install mysql

should handle it, more or less.  But since Leopard (the client version) does not have mysql, you will have had to install that on your own (I installed the MySQL binary package for 10.4, from dev.mysql.com, because there was no 10.5 package available the day after I Leopard was released), and ‘gem’ isn’t good at guessing where it is.  So I need to say

sudo env ARCHFLAGS='-arch i386' gem install mysql -- --with-mysql-dir=/usr/local/mysql

But for some reason that is not quite enough either.  I get this error:

/Library/Ruby/Gems/1.8/gems/mysql-2.7/lib/mysql.bundle:
dlopen(/Library/Ruby/Gems/1.8/gems/mysql-2.7/lib/mysql.bundle, 9):
Library not loaded: /usr/local/mysql/lib/mysql/libmysqlclient.15.dylib (LoadError)

  Referenced from: /Library/Ruby/Gems/1.8/gems/mysql-2.7/lib/mysql.bundle

  Reason: image not found - /Library/Ruby/Gems/1.8/gems/mysql-2.7/lib/mysql.bundle
      from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:32:in `require'
      from my_script.rb:6

Hey, look at that.  It actually tries to load the lib from `/usr/local/mysql/lib/mysql` rather than `/usr/local/mysql/lib`.  Why?  I don’t know.  I couldn’t be bothered to dive into the code, so I went for the quick workaround:

sudo ln -s . /usr/local/mysql/lib/mysql

Silly, and perhaps not the cleanest way to do it, but it works for me.

Not so.  But to make a long story short and save myself some time next time around, here are the mistakes I made:

Private DNS

My wardrobe server (on my home network) hosts a number of public-facing web servers; little things that I set up for friends before I started doing web work professionally.  So, for instance www.norbusam.org points to my fixed home IP, and all traffic on port 80 is forwarded to my server.  The WGR614 that I use for NAT and routing is not smart enough to let me use the public interface from the private part of the network, i.e. if I sit at my desk at home and enter www.norbusam.org into my web browser, I won’t get anywhere.  That would have annoyed me if I hadn’t liked the solution so much: I set up a DNS daemon on the home server and let it trick all home computers to use the internal interface (192.168.x.y) for norbusam.org and others.  Easy as pie.

This came back to bite me for hours on end last night.  notetoself.vrensk.com was pointing at my home server, so while I was hacking the Apache settings on this server and reloading my browser furiously, all I got was my home server.  And since both run Apache, I never got suspicious—it just looked like my changes didn’t take.  Ouch.

SELinux

This server is running Security Enhanced Linux out of the box.  I asked for a vanilla install of FC6, and apparently SELinux is switched on by default.  I decided to keep it on when I got the server since I saw it as a learning opportunity.  It caused me some problems when I set up Postfix when the server was new, but I sorted it out and haven’t thought about it since then.

This, too, came back to bite me for a couple of hours.  It turns out that the default setting of SE for Apache are somewhere between frugal and paranoid.  There are various security bits to be set or cleared, and the default setting assumes that I only want to publish pages from a root-blessed directory and that I don’t want to run any scripts, use a database or have httpd talk to any other servers out there.  Rather than go through all these settings one by one and reverse them (and risk disabling something that was actually permissive to start with) I decided to turn off SELinux for Apache:

# setsebool -P httpd_disable_trans 1

Followed by a restart (not reload) of Apache.  I’m indebted to an article at Begin Linux for this solution.  I can’t recommend reading the actual article though, as it is just a long recapitulation of man pages and other documentation without a trace of explanation or even a try to put things in context.

Oh well.  Next time I will make sure to double check my IPs and disable SELinux at least while setting up a system.

That being said, I might produce the occasional rant or Instruction For Your Betterment or something else intended for public consumption.  I will let you know.