Jun 26, 2013 - Add Google Reader Feeds to rss2email

I’ve been experimenting with rss2email as a replacement for Google Reader. Here’s some ugly bash/python to use to get your XML feeds imported.

Get subscriptions.xml from your Reader dump available at Google Takeout, then put it in /tmp.

for feed in $(python -c "exec('from BeautifulSoup \
import BeautifulSoup as s\nfor f in [f[\"xmlurl\"]\
for f in s(open(\"/tmp/subscriptions.xml\")).findAl\
l(\"outline\") if f.has_key(\"xmlurl\")]:\n\tprint \
f')"); do r2e add $feed; done

Jun 24, 2013 - Legal Television and Movies for Expats

There’s not a lot of original television available in German. IMHO, only Tatort is worth watching. American shows like The Big Bang Theory are broadcast on TV, but I can’t bring myself to watch Sheldon dubbed in German a year after it airs in the United States. Neither can most expats, and more than a few of the non-techies have asked me how to get all their favorite content, so here’s a blog post I can link to.

The simplest solution is to get a VPN and you can watch via various legal streaming services. There’s illegal options like Torrents, but torrents are problematic in Germany, so avoid them. Usenet worked for a while, but that is also troublesome because the biggest indexing site shut down, and for the ones that remain, many TV shows are being removed within minutes of being posted online.

I have money to spend on entertainment, but these companies will not take it. What am I supposed to do?

The last few years felt a lot like the early days of music sharing. Bandwidth is constantly increasing, and now we can share full HD quality video easily. The media companies are going insane and suing file sharers, taking down web sites, and decrying the end of Hollywood and the end of our favorite TV shows and movies. The music problem is largely solved by Spotify, other streaming services, iTunes, and other music stores. Pirating music still happens, of course, but it’s less prevalent.

The entertainment industry is finally starting to apply these models to movie and TV content, which makes watching American content abroad a bit easier.

The first thing any Expat needs who wants to go legal is a VPN. For the non-technical, this is a service you pay for that makes your computer appear as if you’re in the United States, bypassing any geographical restrictions. Yes, the key thing to learn is: you will need to jump through hoops to get legal (or more legal, anyway). I use StrongVPN, and I’m quite happy with the quality of service. Other people use HideMyAss, or any number of other services. After signing up, whatever service should give you instructions to setup your VPN. It’s easier than you think.

Bonus: now that you have a VPN, you can avoid those very sad looking messages from GEMA forbidding you to watch certain YouTube videos.

The second thing is probably an American credit (or debit) card. I have one so that’s not an issue. If you’re not American, or no longer have American accounts, then you might consider trying to get one of those prepaid accounts. There is a workaround for Amazon, though.

What are the legal options?

  • Love Film is the German version of Amazon streaming. No VPN required, but the vast majority of streaming doesn't offer subtitles, nor do they offer English versions. It might have limited utility if you sign up for the DVD's by mail, as many of the DVD's DO come with English subtitles and English audio.
  • iTunes requires no VPN access to use the U.S. store, but tends to be more expensive than Amazon.
  • Amazon Instant Video is Amazon's video streaming service. Amazon Prime members for $79/year get unlimited Prime videos, and free shipping domestically in the United States. Prime videos is a smaller and somewhat different selection than Netflix. There's enough videos to be entertained. Amazon shines in offering the TV shows and New Release movies I can't find on other services. New episodes of TV shows cost around $1.99, and movies are around $2.99 to rent. This is likely something I'll be using quite a bit, as it has the widest selection and often the freshest content. Note: If you don't have an American credit card, you *can* buy an Amazon gift card in dollars using your foreign card, and then use this as your payment for watching video streams. You'll still need the VPN, though.
  • Netflix is great, and I've had it since I moved, because it only costs $8 per month for unlimited content, which tends to be older TV and movies. The newer original content is making Netflix a much better value. Unfortunately, like most services, it's geographically restricted and you need to use a VPN service. There's possible DNS hacks as well which avoid the VPN overhead.
  • Hulu has limited free streaming options which work well but they also offer Plus, which seems like a good idea: also $8/month for unlimited streaming with some commercials. It has quite a few shows that I like to watch, but not all of them. Hulu also aggregates videos from some of the major networks, which is nice, except these don't work if you decide to get an all-in-one device like a Roku. Some shows, like White Collar and other USA Network shows, can have the newest episode delayed by 30 days. VPN and American Credit card required.
  • Google Play is a newcomer to the scene, and I know nothing about them. I'll have to investigate further and maybe do a comparison with Amazon.

You can see from the list above that there’s any number of services to use, and not a single one has all of the TV shown you might want to watch. Even using all of the services above, you will not have access to everything that airs in the U.S. New movies are also hard to come by, with Amazon beating out the other services for the newest selection.

The issue you may be wondering is, what would the cost be to replace your illegal downloading with legal avenues? It varies depending on content consumed. If you’re only relying on Hulu, then it’s quite cheap at $8/month. If you’re paying for Netflix, and buying additional shows on top, it could easily come close to a normal cable bill – or far exceed it.

The situation is improving, but for many pirating still offers a better value. One benefit to pirating is that you “own” the content for as long as you have the bits. Streaming is subject to the whims of executives in Hollywood. What if you want to watch Stargate or Star Trek, for example? Over the last couple of years, it’s been removed and re-added countless times to Hulu, Amazon, and Netflix depending on deals and pricing from the studios. It’s a toss up if content will be available from one day to the next; and of course, you don’t own anything. You can’t even watch offline in most circumstances (Amazon mobile client is one notable exception).

However, for expats wary of delving into the world of pirating, there’s finally enough legal options to stay connected to home.

 

Happy streaming.

May 19, 2013 - Server Provisioning with Puppet ENC

The Puppet External Node Classifier (ENC) is extremely useful for configuring systems based on data in an external source like a CMDB. It is really the killer feature of Puppet, IMHO. Essentially, the ENC is any arbitrary script that Puppet calls to find out which classes and parameters a system should have. The output it expects is in a human-readable serialized data format known as YAML.

One of the key ideas I try to communicate to my consulting customers is the importance of having “one source of truth.” It’s harder than you think with many organizations relying on a number of different databases for information about their systems – or having none of this data tracked at all. It’s rare I find a customer who has just one well-organized source. That being said, I believe the ENC is a good starting point for this one source of truth, because the serialized data that puppet uses can contain other arbitrary information, and used in other automation projects – such as your provisioning layer.

ENC

Ultimately, the backend doesn’t matter too much. I’ve used various different approaches depending on the maturity level of the customer. Sometimes I need to pull the data from more than one source (for example, IP info from an IPAM, and the rest from another source), but once the integration is done, you’re left with at least a single entry point for any of your automation. Projects like the Foreman are rapidly maturing to really replace the need to manually glue these things all together, but rolling your own is a good solution to start from.

Once the ENC script is written, you now need consumers of this information. One of the first things you might like to do is actually start building these systems.

My solution for this is the cobbler-puppet ENC consumer, available on my github here: https://github.com/stbenjam/cobbler-puppet. It’s a fully functional piece of software with RPM spec files. Once you have an ENC, and a Cobbler infrastructure, you can link the two quite easily.

By default the script expects ENC output like this:

---
classes:
  foo:
  bar:
  baz:
parameters:
  cobbler_system_name: www1
  cobbler_hostname: www1.example.com
  cobbler_profile: default
  cobbler_kernel_opts:
    quiet:
    acpi: off
  cobbler_kernel_opts_post:
    quiet:
    acpi: on
  cobbler_ks_meta:
    potato: true
  cobbler_name_servers: [ "8.8.8.8", "8.8.4.4" ]
  cobbler_name_servers_search: example.com
  cobbler_interfaces:
    eth0:
      bonding: slave
      bonding_master: bond0
      macaddress: DE:AD:DE:AD:BE:ED
    eth1:
      bonding: slave
      bonding_master: bond0
      macaddress: DE:AD:DE:AD:BE:EF
    bond0:
      bonding: master
      bonding_opts: "mode=active-backup miimon=100"
      static: true
      ipaddress: 192.168.1.100
      subnet: 255.255.255.0
      gateway: 192.168.1.1

However, you can easily change the “schema” that it expects by modifying enc_parser.py. The entirety of the ENC output is stored as a dict in self._enc, and there is a method that can you change for each information item.

The full documentation is on the github page, but to import a system into cobbler, it’s as simple as running one command:

cobbler-import-enc -s www1.example.com

There are also options for bulk-importing many servers at once, which could be stored in a cron job.

A few people have asked why I don’t just manage the cobbler server using a puppet provider directly, and there’s a few reasons. I did try this, and the simplest reason is that my ruby coding skills are simply not up to par yet. Second, the way I would envision this working is that the individual server configurations would be included in the Cobbler server’s ENC output. The problem with this is it could be extremely large, and Puppet would be a lot slower once you get to adding 5,000+ system records in Cobbler.