Identity. Who or what is doing this thing?

The problem is not just that agents need fewer permissions. It’s that they need legible, revocable, delegated identities of their own. A separate account is not the solution, but it is the least-bad approximation we have today.

The progression

Most people’s journey with AI agents follows a predictable arc. You start by copying and pasting from a chat window. Then you’re babysitting a local agent, approving every action. Then it’s sandboxed in a container, scoped to one repo, running tests on its own. Then one night you go to bed and let it noodle on a problem unsupervised.

At each step, the agent does more under your name. Complex changes still need nudging, but more and more I’m emboldened to let it run on its own. If it’s still using your credentials, every action is attributed to you, and you might not know what happened until morning. Or worse, Claude found the JWT token in ~/Downloads for your production BigQuery database and helpfully ran a query that dropped the wrong table. Least privilege matters. My agents live in their own bubbles, with access scoped so mistakes are recoverable and attributable.

But even with proper sandboxing, identity breaks. Operationally, the agent behaves less like an extension of my hands and more like an independent actor.

A note on autonomy: more autonomy is not always better. Unconstrained agents that open PRs to community repositories without human oversight create slop and place heavy burdens on open source maintainers who are already stretched thin. My agents push to forks and branches. They never open a PR to a community project until I’m ready to review it myself. Giving an agent its own identity doesn’t mean giving it free rein.

Giving my agents their own lives

I run agents in two different contexts, work and personal, and I’ve set up separate identities for each.

At work, I maintain a “not-me” GitHub account for running Claude autonomously in a dedicated environment. It’s completely unprivileged — no write access to any repository where I’m a maintainer or have elevated rights. It can only push to its own forks. It can’t act as me.

At home, OpenClaw has an even more complete identity. Its own email address. Its own GitHub account. Its own Google account. Its own Amazon account. It’s a member of our Google Family, so it can add events to the shared calendar, send reminders, update shopping lists, interact with household systems, and manage mundane logistics. When it sends an email, it sends it as itself. When it pushes code to a personal project, it does so under its own name. It can even make purchases with link-cli, though that one requires me to approve the transaction first. I hope Stripe makes this more configurable. I’d be fine with small purchases to known vendors going through automatically, like a breakfast order or replacing the furnace filters when they’re due.

Two agents, two contexts, two distinct identities, neither of which is mine.

This isn’t just a security measure. It’s an identity measure. These agents are distinct actors in the world, and the systems they interact with should reflect that.

What’s missing

Agents are getting more autonomous, and the platforms haven’t caught up.

There are proposals for agent-specific identity standards, but none have gained traction, and there’s no general agreement on how it should be done. The existing primitives like GitHub Apps and OAuth service accounts have plenty of adoption, but they were designed for CI systems and developer integrations. Setting up an agent with Google, for example, means creating a project in Google Cloud, configuring OAuth consent screens, managing token refresh. Why does it have to be that complicated? Giving agents their own user accounts makes sense to an extent, since it’s the closest approximation we have, but none of these platforms have a concept for “this account is operated by an AI agent on behalf of a specific human” with the right affordances: visible provenance, scoped authority, delegation, approval policies, audit logs, and independent suspension.

The world is not built for agents yet. When Samai orders my breakfast, it struggles every time with the order pages. Amazon is notoriously difficult to navigate with computer use. Sites throw up captchas when behavior looks suspicious, because they can’t distinguish “authorized agent acting for a human” from “bot abuse.” Everything from the UIs to the identity systems assumes a human is on the other end.

Right now I’m stitching agent identity together with regular user accounts, careful naming, and scoped permissions. It works, but it’s a workaround. A real system would let me say: this agent is operated by Stephen, can act only within these scopes, and must ask for approval above these thresholds. I think proper agent identity, permissions, and interfaces designed for their capabilities are coming soon. Until then, this is the best we’ve got.

It is a subclass of “have an LLM make adhoc tools for me”.

I am probably not the first person to invent this pattern, but I have found myself coming back to it for several more problems with a lot of success.

The Problem

Out of a couple hundred thousand CI job runs, I had a corpus that I believed were generic “infrastructure” problems, either our own internal build infrastructure or issues in external providers. These are jobs that never reached the point of doing useful work (producing test results). I wanted to screen them and understand where the problems were coming from and if they were our fault, spend time focus on fixing the most common.

Reading 10,000 logs manually? Not happening. Regex pattern matching? Well, I’d have to know what I’m looking for first. Pure LLM classification? A good starting point, but it doesn’t have the expertise I have.

The Pattern

The Screener Loop has five phases, with the feedback loop repeating until the results are good enough.

Phase 1: Bulk LLM Classification

Split the dataset into batches and send each to an LLM subagent in parallel. Each agent reads ~100 logs and returns structured classifications:

{
  "2034253746107584512": {
    "category": "lease failure",
    "snippet": "failed to acquire lease for aws-3-quota-slice"
  }
}

I used 100 parallel subagents to classify all 10,000 logs. The LLM doesn’t need perfect instructions. Give it a rough category list and let it create new categories when it finds something unexpected. This first pass gets you maybe 60-70% accuracy, which sounds bad until you realize it just turned an impossible manual task into a tractable review task.

Phase 2: Materialize Results as an Interactive Tool

This is what makes the pattern work. Don’t just dump classifications into a spreadsheet. Instruct the LLM to create a self-contained HTML screener tool that lets the reviewer:

• Browse every classification with the original log right there
• Change categories with a dropdown
• Add freeform notes explaining why something is wrong
• Filter by category, job name, or review status
• See everything persisted in localStorage so they can close the tab and come back

The screener is a single HTML file with all data embedded as JavaScript constants. No server needed, just open it in a browser. The screener tool defaults to a randomized the order, so you are seeing a diversity of data.

Phase 3: Human Review and Correction

The human reviews a sample (maybe 30-50 items) and does two things:

1. Recategorizes misclassified items using the dropdown
2. Writes notes explaining the pattern they see

The notes are the important part. Instead of just changing a category, the reviewer writes things like:

"'0/121 nodes are available: 23 Insufficient memory, 59
Insufficient cpu', this is a capacity issue on the
build cluster"

"Back-off pulling image '.*stable:test', this is
a missing test container image"

These notes contain the pattern, not just the correction. When exported as JSON, they become instructions for the next phase.

Phase 4: Pattern Generalization

I then give the JSON file back to the LLM in the same session: take the human’s corrections, generalize them, and re-evaluate all other entries.

One human correction often fixed hundreds of items.

Phase 5: Regenerate and Repeat

Regenerate the screener with updated classifications and go back to Phase 3. Each iteration the “unknown” bucket shrinks and the meaningful categories grow. I went from several thousand unknowns to under 500 in three review cycles, with about a dozen distinct categories emerging.

After completing the screener cycle, the LLM can produce a static report - document - markdown, HTML, etc. A reccomendation from a coworker had me introduce a heatmap, which was incredibly useful to pinpoint one-off vs clustered vs perennial problems.

Augmenting with K-Means Clustering

After the first few review cycles, I still had thousands of “unknown” logs. Reviewing them one by one was slow. I needed a way to find structure in the unknowns, without spending a ton of money on LLM tokens.

TF-IDF vectorization + K-means clustering turned out to be the right tool. I only have a vague idea of what this means, I did poorly in my undergrad stats course. I might be wildly misapplying the concepts, but it did work quite well.

This section is written by the LLM who described the approach. You can skip over it, and just know “the LLM came up with a way to group like things”.

1. Extract only error lines from each log by filtering to lines containing keywords like error, fail, timeout, refused, denied, insufficient. This eliminates the noise of normal log output.

2. Normalize aggressively by stripping ANSI codes, timestamps, dates, IP addresses, SHA256 hashes, CI-specific namespaces, and job name patterns. Without this, clusters form around “ran on March 17” instead of “connection refused.”

3. Vectorize with TF-IDF using bigrams (1-2 word combinations), then run K-means with cluster count proportional to dataset size (max(5, min(40, len(docs) // 30))).

4. Use chi-squared discriminative terms instead of centroid-nearest terms. Standard K-means gives you the terms closest to each cluster centroid, which are often generic (“error”, “failed”, “pod”). Chi-squared testing finds terms that are statistically over-represented in each cluster compared to all other clusters. The difference is dramatic:

   • Centroid terms: step failed, pod failed, time error, error component
   • Chi-squared terms: ocp failed, containernotfound, specified container

   The second set actually tells you what’s distinctive about the cluster.

5. Integrate clusters into the screener. Each item gets purple cluster terms in the sidebar, and a “View all in cluster” button that filters to just that group.

This let the reviewer think in terms of groups. Instead of “let me look at thousands of unknowns one at a time,” it became “cluster #17 has 71 jobs with terms ‘stricthostkeychecking, userknownhostsfile, connecttimeout’, those are all SSH config failures during install.” One click to view the cluster, confirm a few samples, create the category, and pattern-match the rest.

Some clusters surfaced brand new problems I didn’t know were occuring.

When to Use This Pattern

The Screener Loop works when you have:

• Hundreds to tens of thousands of unstructured artifacts to classify
• Domain expertise that can’t be fully encoded upfront because you know it when you see it, but can’t write all the rules in advance
• Categories that emerge from the data rather than being defined beforehand

It’s particularly effective for incident analysis, log triage, support ticket categorization, and any situation where you’re trying to find patterns in a large corpus of messy text.

NanoClaw was appealing for its simplicity, but it has a heavy Ubuntu focus and is tightly coupled to Claude Code. I love Claude Code. I use it constantly, and it has the best plugin ecosystem of any AI tool I’ve used. But being locked to one vendor makes me uneasy.

OpenClaw is what I liked the most. It’s big. It’s sprawling. I’m mildly worried it’ll go off the rails at some point. But with cautious use, it’s been remarkably capable. Here’s what a week with it actually looked like.

Meet Syl

My assistant is named Sylphrena, Syl for short, an Honorspren from Brandon Sanderson’s Stormlight Archive. She wakes up fresh every session. Continuity comes from a set of files in her workspace: SOUL.md (personality), USER.md (who I am), MEMORY.md (long-term learnings), and daily memory logs. It sounds simple. It works surprisingly well.

The Heartbeat Loop

Every 30 minutes, Syl runs a heartbeat. I define what she checks in a HEARTBEAT.md file: email, package tracking, stock quotes, calendar, whatever I want monitored. She runs through the list, decides what’s actually worth my attention, and pings me on Telegram only when something matters. Everything else gets logged quietly.

She caught a NAS failure at 4 AM. She notified me when my USPS package arrived at my home post office. She gave me a heads-up when NVDA dropped past -4.75% pre-market. She also automatically picks up new tracking numbers from shipment notification emails. The stuff that doesn’t clear her bar? I never hear about it.

MailGuard: A Sane Email Layer

Email is a potent attack vector. Pulling untrusted content into an agent’s context window is dangerous. So I built MailGuard MCP, a local mail proxy that only shows full message contents from explicitly trusted senders. Everyone else appears as <untrusted_sender> with just their address. Claude Code had the whole thing ready in about 15 minutes.

The MCP runs isolated in its own agent with no access to other tools. Frontier models were already pretty good at catching prompt injection on their own (Syl called my own prompt injection attempt “garbage”), but I like the belt-and-suspenders approach.

Skills

OpenClaw uses a skills system where each skill is a directory with a markdown file describing what it does and how to use it. I set up skills for stock quotes, a daily morning briefing, automatic download of newly available library holds from Libby, and AMC movie booking.

The AMC skill was the most fun. It drives a Chrome browser on its own. I asked Syl to book me a ticket for Project Hail Mary, she picked a seat, described it, and asked for approval. I changed my mind on the showtime halfway through, so she canceled, refunded, and rebooked. The skill was written during that session, based on what she learned.

Things Don’t Always Go Smoothly

This is not magic. I asked Syl to warm up my car via Home Assistant and she struggled for several minutes before I realized my Tesla Fleet API token had expired. Booking dinner via Resy didn’t work either. I suspect they have some bot detection mechanism (the search bar appeared functional but silently did nothing). I didn’t dig into it.

Do I Really Need an LLM for This?

Let’s be real: most of what the heartbeat does could be a shell script. Fetch a stock price, check a tracking number, send a Telegram message. None of that requires a language model. I could wire it up with curl and jq in an afternoon and burn zero tokens.

But the value isn’t in any single task. It’s in the glue. Syl decides whether something is worth telling me about. She compares today’s stock snapshot to yesterday’s and writes a different kind of message depending on how bad it is. She reads an email subject line and decides it can wait. She notices that a package tracking status changed from “in transit” to “out for delivery” and flags it, but ignores the five intermediate scans that don’t matter.

That judgment layer is what makes it feel like an assistant instead of a cron job. Could I hand-code all those heuristics? Sure. But then I’m maintaining a pile of brittle if-statements that break the first time the world does something I didn’t anticipate. The LLM handles the long tail for a few cents a run.

Models and Costs

An always-on agent burns tokens fast. I nearly exhausted my OpenAI Codex budget ($20 plan) in two days, and my Anthropic plan is close behind. For simple scheduled tasks, Qwen 3 8B running locally does fine. For the heavier stuff, Ollama Cloud’s GLM-5 seems like a great deal and would probably get me through the month.

I also spent a fair amount of time on context management, stripping out a lot of what OpenClaw includes by default. It is not nearly as efficient as Claude Code’s context handling, and the extra tokens add up when you’re running heartbeats every 30 minutes.

What’s Next

Syl has been running for a week and the setup already feels essential.

The agentic future is messy, risky, expensive, and not always reliable. It’s also already here, and it’s useful enough that I’m not turning it off.

Syl helped write this post. I asked her to review everything we did together over the past week in OpenClaw and pull out the highlights. She drafted sections, I rewrote them, and we went back and forth until it read right. She approved the final version. 🌬️

$ tar xvf tofu-wonder.tar.gz
.config/
.config/server.xml
.config/database.xml
README.txt
LICENSE.txt
tofu-wonder
001.dat
002.dat
003.dat
004.dat
005.dat
[...]
943.dat

You just got tarbombed. In older versions of tar, tarballs could even contain absolute paths and potentially overwrite existing files on your file system. These days, most versions of tar prevent this unless explicitly allowed, so the worst that happens is a particular tar archive litters it’s files in whatever unfortunate directory you were in when you extracted it. Have fun cleaning that up.

Ok - so how to avoid it? I now include this line in my .zshrc:

export TAR_OPTIONS="--one-top-level"

This option extracts all files into a directory named by the basename. In the example above, it’d now look like this:

$ tar xvf tofu-wonder.tar.gz
tofu-wonder/.config/
tofu-wonder/.config/server.xml
tofu-wonder/.config/database.xml
tofu-wonder/README.txt
tofu-wonder/LICENSE.txt
tofu-wonder/tofu-wonder
tofu-wonder/001.dat
tofu-wonder/002.dat
tofu-wonder/003.dat
tofu-wonder/004.dat
tofu-wonder/005.dat
[...]
tofu-wonder/943.dat

Perfect! But, it’s better not to make users do this. The first way to prevent this is to include the top-level directory when you’re creating a tarball:

tar czvf tofu-wonder.tar.gz tofu-wonder/

Another option is to use transform and replace . with something else:

tar czvf tofu-wonder.tar.gz --transform "s?^\.?tofu-wonder-0.1.1?"  .

Lukáš from the Foreman project proposed an RFC to enable this functionality in Foreman. Much of this is now implemented: Foreman has an HTTPBoot Smart Proxy module that serves the TFTP boot directory via HTTP, and makes Foreman aware of various DHCP settings. There are still some issues to be resolved before this is ready for users to use.

This blog post is mostly my notes from us researching how HTTP boot works, how grub2 supports HTTP boot, and how to test with libvirt. We used the edk2 firmware for QEMU/KVM, although much of these notes are generally applicable to hardware as well - we’ve tested on at least one real world baremetal server and was able to provision end-to-end using HTTPS.

Configure libvirt

Out of the box, QEMU will use BIOS. However, you can install the Tianocore firmware to get UEFI. This package is called edk2-ovmf on Fedora.

If you are on CentOS 7 or want to use the latest nightlies, you can get them from this fedora documentation. Last I checked, they don’t have TLS support compiled, which means you can’t enroll a TLS certificate to make HTTPS boot work. The Fedora firmware does support this.

After installing the firmware package on CentOS, you’ll also need to configure the nvram setting in /etc/libvirt/qemu.conf. Newer Fedoras are already aware of and will look in this path for firmwares:

nvram = [
  "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd"
]

Once you do that, you can create an UEFI VM, by selecting a UEFI x86_64 firmware:

Configure DHCP

Your DHCP configuration must be aware of HTTP clients in order to set the filename to an URL. The relevant snippet from my own DHCP config is below. It’s important to set the vendor-class-identifier as HTTPClient, otherwise your host will not use the filename as an HTTP URL.

option arch code 93 = unsigned integer 16; # RFC4578

# This is for UEFI HTTP:
class "httpclients" {
  match if substring (option vendor-class-identifier, 0, 10) = "HTTPClient";
  log(info, "HTTP UEFI Client Detected");
  option vendor-class-identifier "HTTPClient";

  if option arch = 00:0F {
    filename "http://example.com/bootia32.efi";
  } else if option arch = 00:10 {
    filename "http://example.com/bootx64.efi";
  }
}

Boot loader

I’ve tested HTTP boot with both iPXE and grub2. If iPXE supports your network card, you might consider using it. It supports UEFI HTTP Boot well.

If you want to use grub2, hang on to your hat - there’s a number of bugs in any of the latest shipped versions, including in Fedora 30. Fixes that enable using relative paths will ship in Fedora 31. The bug for that is here.

If your grub2 configuration uses fully qualified paths in all places, you won’t need this patch, but you won’t be able to use your grub2 configuration for both legacy TFTP and HTTP clients.

Enrolling a CA Certificate

For libvirt, I created a VFAT image, and stored a copy of my CA certificate there:

$ dd if=/dev/zero of=/tmp/ca.img bs=1440K count=1
1+0 records in
1+0 records out
1474560 bytes (1.5 MB, 1.4 MiB) copied, 0.000856816 s, 1.7 GB/s
$ mkfs.vfat /tmp/ca.img
mkfs.fat 4.1 (2017-01-24)
$ sudo mount -o loop /tmp/ca.img /tmp/mnt
$ sudo cp /tmp/ca/ca.crt /tmp/mnt
$ sudo umount /tmp/mnt
$ sudo cp /tmp/ca.img /var/lib/libvirt/images

I then attached it to libvirt:

and enrolled the certificate in the Device Manager menu:

Assuming your DHCP configuration is setup correctly, then you can select HTTP boot from the Boot Manager, or reboot the host, and you will boot via HTTPS.

I decided to opt for a local program, and narrowed down my choices between Tuft’s part-time M.S. in Computer Science, and Harvard Extension’s awkwardly named Master of Liberal Arts in extension studies, field: Software Engineering. There’s a difference between computer science and software engineering, of course, but the Harvard ALM was flexible enough that I could include the theoretical stuff I wanted and it was easier to get to Harvard on public transportation. The degrees from the extension school seem to be awkwardly named to differentiate the Extension school from the other more traditional schools at Harvard. Still, I don’t get why it’s a Master of Liberal Arts, that’s not really reflective of the coursework. There’s been some effort to change that, although I doubt I’ll ever see it as I’m nearly done.

I’ve mostly taken one course at a time, but this year my awesome employer graciously let me take a leave of absence to study full time to wrap things up quickly as my GI Bill benefits expire soon.

Update: I’ve since returned from my LoA, and finished all but my last course (the Capstone). I’ve updated this blog post with reviews of everything I’ve taken so far, and some conclusions I’ve made so far about the ALM.

Courses

• CSCI E-97: Software Design Patterns
• CSCI E-93: Computer Architecture
• CSCI E-95: Compiler Design
• CSCI E-92: Operating Systems
• CSCI E-28: Unix Programming
• STAT E-100: Intro to Statistics
• PHYS S-123: Laboratory Electronics: Analog and Digital Circuit Design
• CSCI E-55: Java, Hadoop, Lambda Expressions, and Streams
• CSCI E-15: Dynamic Web Design
• CSCI E-134: Networks
• CSCI E-48: Secure Mobile Computing
• CSCI E-599: Capstone

CSCI E-97: Software Design Patterns

This class was a deep dive into software design patterns. It taught most of the key patterns from the “Gang of Four.” It was also a heavy deep dive into UML.

I thought the class was useful, but the material was dry. It is a required part of the ALM.

CSCI E-93: Computer Architecture

This and the following two classes (CSCI E-95, and E-92) were my best experience at Harvard so far. I used this class as my “theoretical foundations” requirement for the 3 admissions courses.

In this class, I built a computer processor from scratch. I designed an instruction set, wrote an assembler, an emulator, and then finally implemented the processor using VHDL that ran on a physical piece of hardware (an FPGA development board from Altera). My final project video is on YouTube.

This and the other 2 classes are designed such that if you work through each of the problem sets, you’ll get a working thing in the end. Out of all 3 classes, this one had more freedom as you could really design any kind of processor you wanted. Some people tried more adventurous things like stack machines or getting pipelining working, but this was my first experience at this layer so I ended up doing a 16-bit, mostly MIPS-like architecture. My final project’s special feature was an LFSR. Many students opt for interrupts or hardware multipliers as theirs.

If you end up taking this, go to section. It’s extremely useful and a lot of implementation suggestions are given.

CSCI E-95: Compiler Design

I wrote a compiler for a large subset of C using C along with classic compiler tools (Flex and Bison). Whereas many other universities teach classes where you only learn theory, or only implement a “simple” language - this class stood out in that you literally write a compiler for C and have to understand all of its quirks.

It’s essentially C89 minus structs, unions, and function pointers. The final project is implementing an optimization stage in the compiler, mostly adding simpler peephole optimizations on basic blocks. Many students also work on more advanced register allocation strategies like graph coloring.

My final project video is here.

It’s worth noting, I didn’t know even know C when I started the class, but I knew it pretty well by the end.

CSCI E-92: Operating Systems

In this class, we learned the important operating system concepts, and then implemented an OS on a Freescale K70 Tower. You start out writing a small shell, your own implementation of malloc, various system calls in the OS, and towards the end of the class everything comes together when you write own scheduler and get multitasking working.

My final project was a rather complete POSIX-like permissions system, and interrupt-based Semaphores. I also got multiple serial ports working on the device, which made the demos a little more interesting.

My final project video is here.

CSCI E-28: Unix Programming

This class dives into the details of how POSIX systems programming works (and more specifically, the class touches on a lot of Linux-specific things). I took this concurrently with CSCI E-92, which it was a nice complement for. I’d reccomend taking this first, or also concurrently, to see how real operating systems design their system calls.

During this class, I wrote a shell (a bit more complex than the one I wrote for CSCI E-92), a pong game using curses, as well as a multithreaded web server for my final project. I wouldn’t say this is a particularly demanding course if you’re already familiar with Unix-like operating systems and know C. Most of the problem sets come with significant starter code.

The class is taught by the author of Understanding UNIX/LINUX Programming.

STAT E-100: Intro to Statistics

This class was underwhelming. I was hoping for a deeper dive into R, but it was much more using R as a REPL with 99% of the R code given to us. It provided a good introduction to statistical concepts, but was rather shallow in the depth of the topics that were covered.

I also took this as an online-only class, as it was the only stat class that’d fit into my schedule. The video lectures were really great, however the problem sets were not very challenging and mostly multiple choice. I expected it to be a little harder, and was disappointed I blew one of my elective slots on this class considering I could’ve got this out of Khan Academy on my own.

There’s a few other statistics classes at Harvard Extension (100, 102, 110, etc), offered by a number of different instructors. Perhaps some are better than others.

PHYS S-123: Laboratory Electronics: Analog and Digital Circuit Design

This was an 8-credit (2 course) summer program that ran Monday through Thursday, 9 to 1pm (officially) over seven weeks. On days with labs, the time was more realistically 2:30 or 3pm. Add on homework and study time, I was getting home in the evening nearly every day.

Typically, this is taught as two separate classes: a semester on analog electronics and a semester on digital electronics. This summer school version is intense - covering this amount of material in 7 weeks is daunting, and for the summer course some of the more interesting things are removed. Instead of the “big board” path where you build up your own computer on breadboards, we worked with a SiLabs microcontroller that had most everything built-in.

Still, it was a great experience and I’m glad I took this class. The analog section starts off covering voltage, current, and resistance and building passive circuits. It moves on to transistors, both BJT and MOSFETs, and you go on to build an op amp from discrete parts to understand what’s inside. It goes on to cover op amps in detail covering usages of positive and negative feedback, and on the final day of the analog part of the course, we designed and built a group project that transmitted and received an audio signal using infrared light.

The digital part of the class starts off with boolean logic, HDLs, and logic gates. CSCI E-93 covered a lot of this, but the electronics were abstracted away from us in VHDL. In this course, you look at what’s actually inside both TTL and CMOS logic gates, build analog-to-digital or digital-to-analog converters from parts, design and build state machines using flip flops, etc. The final week of the class is working with the SiLabs microcontroller, and writing assembly programs for it.

The text book we used is The Art of Electronics, along with the accompanying student lab manual.

CSCI E-55: Java, Hadoop, Lambda Expressions, and Streams

This met the ‘cloud’ requirement in the ALM program – because of the short section on Hadoop. It was the only available class that met the cloud requirement, so I had to take it. It would be a useful class if you did not know Java, but were already an experienced developer in something else. I did know Java so it was largely review. The overview of Java 8 features, and Hadoop were great, but that only made up about the last 1/4 of the course.

The instructor is great, and provides a history of his experience as a software developer over several decades. That was my favorite part of the class.

CSCI E-15: Dynamic Web Design

Another requirement for the ALM is ‘web design.’ The course was very well organized, and helpful if you knew HTML/CSS and did not have any background with a MVC framework. The course teaches Laravel, PHP, and HTML and students get a handle on git and work a bit with the LAMP stack.

For me, a waste of time and I wish ‘web design’ was not required, and instead given another elective spot.

CSCI E-134: Networks

This course I had intended to fulfill my ‘data communications’ requirement for the ALM. It was an exploration of all kinds of networks, such as social networks. It was marketed as an intersection of economics/computer science. It was run concurrently with the Harvard College version of the class.

I dropped it early on, as the advertised pre-reqs were not correct. One needs a deeper background than I had in math, including linear algebra. I’ve always regretted not having a better background in math. Maybe something I’ll improve after I graduate.

CSCI E-48: Secure Mobile Computing

Instead of E-134, I took this for the ‘data communications’ requirement.

The video lectures were largely reading the text slides word-for-word, unfortunately. Students do get some experience with packet sniffing WiFi networks with kismet and such, and some content related to 3G/LTE mobile networks. However, I didn’t like the course structure. Weekly requirements were forced discussion: watching a video and writing a summary post, and then replying to two other students’ posts.

The course staff was in general great, and I found them knowledgable and the labs OK, but I was hoping for more. Labs were interesting, but not very deep. Had I known CSCI E-134 wasn’t going to work out, I’d have taken one of the internet architecture courses on offer, but they did not fit with my schedule by the time I dropped it.

CSCI E-599: Capstone

Currently in progress for Spring 2018, will update when complete.

Conclusion

I have finished all but the Capstone, which I’m currently taking in Spring 2018.

There are many amazing courses at HES, but the 5 elective slots in the ALM program are not enough, especially when they’ve filled up the requirements with fluff like web design, “cloud”, and data communications where the classes only very loosely fit into those buckets.

I am a little bit disappointed by how many courses I felt like were wasting my time. My own professional goals do not align with how HES has structured the ALM. I’d rather have more elective slots, which I could’ve used to correct my weakness in math (MATH E-23A and others), or get more experience with research (CSCI E-191).

However, overall my experience at HES has been a good one. There are a number of classes that I think were high quality and well worth my time. I wrote a compiler, an operating system more or less from scratch, and designed my own instruction set and processor. I’ve built op amps on breadboards from transistors, and a combinational lock out of push buttons, LED’s, and flip flops. I’ve written a multithreaded web server, my own shell, and my own implementations of a ton of Unix utilities. All of this gave me a better foundation in how computers actually work, something I don’t think I had a good grasp on before I began this journey - even though I had been working as a developer and sysadmin for a number of years.

Two projects that I'm really loving at the moment are The Foreman and FreeIPA. The Foreman is lifecycle management tool for physical and virtual servers (think Cobbler on PCP), and FreeIPA provides central authentication: directory services, kerberos, policy enforcement, and a PKI infrastructure. Why not glue them together? This is my first attempt, and it's all a bit manual and unpolished. There's an effort to get this integration into the Foreman Smart Proxy itself.

All of the scripts here are in a Github Gist.

• Foreman authenticates against FreeIPA
• Signed Certificates for Foreman, Puppetmaster, and Clients
• New Hosts automatically register to IPA and get a signed certificate
• A host gets deleted from IPA when it is deleted from Foreman

Prerequisites

• Installed Foreman Server on a Red Hat-based distro (RHEL, CentOS, Fedora, et al).
• Foreman server is registered to FreeIPA
• Installed FreeIPA (or Red Hat IdM) Server

Contents

1. Foreman LDAP Authentication
2. FreeIPA PKI Infrastructure
3. Automatic IPA Registration/Deletion

1. Create a foreman.ldif file, replacing dc=bitbin,dc=de with your DN, and providing an appropriately secure password:

   dn: uid=foreman,cn=sysaccounts,cn=etc,dc=bitbin,dc=de
   changetype: add
   objectclass: account
   objectclass: simplesecurityobject
   uid: foreman
   userPassword: 8j926SEpcOvM0WLI
   passwordExpirationTime: 20380119031407Z
   nsIdleTimeout: 0

2. Import the LDIF (change localhost to an IPA server if needed), you'll be prompted for your Directory Manager password:

   # ldapmodify -h localhost -p 389 -x -D \
   "cn=Directory Manager" -W -f foreman.ldif

3. Add an IPA group for foreman_users (optional):

   # ipa group-add --desc="Foreman Users" foreman_users

4. Now login to the Foreman as an Admin, click on "LDAP Authentication" under More/Users. Then click New LDAP Source and fill in the details, changing dn's where appropriate to your own domain:
   • Server: astriaporta.bitbin.de
   • Port: 636
   • TLS: checked
   • Account username: uid=foreman,cn=sysaccounts,cn=etc,dc=bitbin,dc=de
   • Account password: 8j926SEpcOvM0WLI
   • Base DN: cn=accounts,dc=bitbin,dc=de
   • Filter (optional): (memberOf=cn=foreman_users,cn=groups,cn=accounts,dc=bitbin,dc=de)
   • Automatically create accounts in the Foreman: checked
   • LDAP mappings are as the examples given.
   </li> </ol>

   FreeIPA Certs for Foreman + Puppet

   End Goal: Foreman and the Puppetmaster use certificates from the FreeIPA server.
   1. Backup the existing SSL directory:

      # mv /var/lib/puppet/ssl /var/lib/puppet/ssl.old

   2. Make the appropriate directory structure:

      # mkdir -p /var/lib/puppet/ssl/{private_keys,certs}

   3. For the next steps you need to have a Kerberos ticket for a user with sufficient privileges (e.g. admin)

      # kinit admin

   4. Create the service principal:

      # ipa service-add puppet/`hostname`

   5. Request certificates from the IPA server

      # ipa-getcert request -K puppet/`hostname` -D `hostname` \
       -k /var/lib/puppet/ssl/private_keys/`hostname`.pem \
       -f /var/lib/puppet/ssl/certs/`hostname`.pem

   6. Check on the request, you should see the status as MONITORING if successful:

      # ipa-getcert list
      Request ID '20131106211000':
          status: MONITORING
          stuck: no
          key pair storage: type=FILE,location='/var/lib/puppet/ssl/private/gatebuilder.bitbin.de.pem'
          certificate: type=FILE,location='/var/lib/puppet/ssl/certs/gatebuilder.bitbin.de.pem'
          CA: IPA
          issuer: CN=Certificate Authority,O=BITBIN.DE
          subject: CN=gatebuilder.bitbin.de,O=BITBIN.DE
          expires: 2015-11-07 21:10:01 UTC
          eku: id-kp-serverAuth,id-kp-clientAuth
          pre-save command:
          post-save command:
          track: yes
          auto-renew: yes

   7. Copy the IPA CA.cert:

      # cp /etc/ipa/ca.crt /var/lib/puppet/ssl/certs/ca.pem

   8. Take a peek in the SSL directories, and you'll see our new certs:

      # ls  {private_keys,certs}
      certs:
      ca.pem  gatebuilder.bitbin.de.pem
      
      private_keys:
      gatebuilder.bitbin.de.pem

   9. Make sure permissions are sensible:

      # chown -R puppet:puppet /var/lib/puppet/ssl
      # chmod 600 /var/lib/puppet/ssl/{private_keys,certs}/`hostname`.pem

   10. Edit /etc/puppet/puppet.conf:
       • Add to [main]:

         # This disables the CRL.  I need to fix this at a
         # later time
         certificate_revocation = false

       • In [master], change:

         ca = false

       • Restart httpd (Foreman-configured Puppet runs in Passenger):

         # service httpd restart

   11. Browse to the Foreman, and you should see it using the new SSL certificates signed by your IPA CA. Ideally you should import the IPA CA.crt on your local box and trust it.
   </ol>

   Registration at Provision-Time

   The idea here is that our machines when foreman creates them are automatically registered to FreeIPA with a one-time password, and if later deleted in the Foreman, they are removed from FreeIPA too. Hosts also get an SSL certificate signed by the FreeIPA server to talk to puppet. The flow looks like this:

   Creating IPA User with Right Permissions

   A previous version of this guide called this user "foreman" - don't do that, it'll interfere with upgrading later, as the RPM packaging expects to use a local user named "foreman."
   1. Create the user:

      # kinit admin
      Password for admin@BITBIN.DE:
      # ipa user-add --first="The" --last="Foreman" foreman_reg \
      --password
      Password:
      Enter Password again to verify:
      --------------------
      Added user "foreman_reg"
      --------------------

   2. Grant host enrollment privileges:

      # ipa role-add-member --users=foreman_reg "Host Enrollment"

   3. We need to modify the Host Enrollment role to actually allow the Foreman user to add brand new hosts and delete them too -- so Foreman can completely manage the machine lifecycle.

      # ipa privilege-add-permission 'Host Enrollment' \
      --permissions='Add Hosts'
      # ipa privilege-add-permission 'Host Enrollment' \
      --permissions='Remove Hosts'

   4. Change foreman password after first time:

      [root@gatebuilder ~]# kinit foreman_reg
      Password for foreman_reg@BITBIN.DE:
      Password expired.  You must change it now.
      Enter new password:
      Enter it again:

      </ol>

      Configuring the Create/Destroy Hook

      Grab the scripts from the Github Gist.
      1. On Foreman, install the hooks gem:

         # yum -y install ruby193-rubygem-foreman_hooks

      2. And make the directory structure we need:

         # mkdir -p /usr/share/foreman/config/hooks\
         /host/managed/{create,destroy,after_commit}

      3. Put foreman-ipa into /etc/sysconfig/ with the right permissions

         # chown foreman /etc/sysconfig/foreman-ipa
         # chmod 600 /etc/sysconfig/foreman-ipa

      4. Configure /etc/sysconfig/foreman-ipa

         # Are we using IPA as the CA?
         CREATE_SERVICE_PRINCIPAL=true
         
         # Allow Foreman to delete hosts from IPA
         PREVENT_DELETING_HOSTS=false
         
         # Hostname of an IPA server
         IPA_SERVER="astriaporta.bitbin.de"
         
         # User with appropriate permissions
         IPA_USER="registration"
         IPA_PASS="password"
         
         # Foreman API User/Password
         FOREMAN_USER="apiuser"
         FOREMAN_PASS="apipass"

      5. Put 10_integrate_freeipa.sh into /usr/share/foreman/config/hooks/host/managed/create and create a symlink to destroy and after_commit:

         # ln -s /usr/share/foreman/config/hooks\
         /host/managed/create/10_integrate_freeipa.sh ../destroy
         # ln -s /usr/share/foreman/config/hooks\
         /host/managed/create/10_integrate_freeipa.sh ../after_commit

      6. Restart foreman to get it to notice the new hooks:

         # service foreman restart
         # service httpd restart

      7. Take a look at the logs to make sure the hooks were registered, look in /var/log/foreman/production.log:

         Finished registering 1 hooks for Host::Managed#destroy
         Finished registering 1 hooks for Host::Managed#after_commit
         Finished registering 1 hooks for Host::Managed#create

      8. The last step in this is integrating into your provisioning template. You'll need to get ipa-client installed in your packages list, and remove the other puppet registration thingy from the Foreman. I have a snippet that looks like this:

         # Register to IPA, two times
         # in case of https://fedorahosted.org/freeipa/ticket/3377
         ipa-client-install --mkhomedir -w <%= @host.params['ipa_onetime'] %> -f -U
         ipa-client-install --mkhomedir -w <%= @host.params['ipa_onetime'] %> -f -U
         
         # Make Puppet Certificate Directories
         mkdir -p /var/lib/puppet/ssl/{private_keys,certs}
         
         # Generate IPA Certificate
         ipa-getcert request -K puppet/<%= @host.name %>  -D <%= @host.name %> \
         -k /var/lib/puppet/ssl/private_keys/<%= @host.name %>.pem \
         -f /var/lib/puppet/ssl/certs/<%= @host.name %>.pem
         
         # Workaround for "stack too deep" problem
         # http://projects.puppetlabs.com/issues/21869
         cp /etc/ipa/ca.crt /var/lib/puppet/ssl/certs/ca.pem
         
         cat <<EOF > /etc/puppet/puppet.conf
         [main]
             # The Puppet log directory.
             # The default value is '$vardir/log'.
             logdir = /var/log/puppet
         
             # Where Puppet PID files are kept.
             # The default value is '$vardir/run'.
             rundir = /var/run/puppet
             ssldir = /var/lib/puppet/ssl
             server = <%= @host.puppetmaster %>
         
         [agent]
             # The file in which puppetd stores a list of the classes
             # associated with the retrieved configuratiion.  Can be loaded in
             # the separate ``puppet`` executable using the ``--loadclasses``
             # option.
             # The default value is '$confdir/classes.txt'.
             classfile = $vardir/classes.txt
         
             # Where puppetd caches the local configuration.  An
             # extension indicating the cache format is added automatically.
             # The default value is '$confdir/localconfig'.
             localconfig = $vardir/localconfig
         
             certificate_revocation = false
             certname = <%= @host.name %>
         EOF
         
         puppet agent --test
         chkconfig puppet on

      9. When the machine boots it will use one-time password authentication with FreeIPA and grab an SSL certificate for use with Puppet. And you get the bonus of when you delete the machine in Foreman, it gets deleted in IPA too.
      </ol>

I’ve just returned from five weeks in Southeast Asia, and I have to say it was one of the best experiences of my life. My time in Asia began with a 10-day Buddhist retreat in the middle of the jungle learning Vipassana.

These courses are free of charge; no money is accepted from any new student. Likewise, no teacher or volunteer receives any payment for their services. Previous students have paid for you to be there. It’s intentionally designed this way, and is this way for a similar reason that Buddhist monks beg: it’s incredibly humbling to realize you eat, sleep, and drink only due to the compassion of others.

The web site is dhamma.org, and you can find a list of centers in many countries (including Europe and the U.S.). After applying to a 10-day course, you’ll hear back in a few days and get a confirmation if there’s space.

I took the course in Thailand, at the Dhamma Kancana center. It’s located in the beautiful Thai jungle, about 6 hours by bus from Bangkok. You can see photos of the center here. It is known as one of the nicest centers in Thailand, but there are several closer to Bangkok if you’d prefer not to atravel far.

What is Vipassana?

From the website:

"Vipassana, which means to see things as they really are, is one of India's most ancient techniques of meditation. It was taught in India more than 2500 years ago as a universal remedy for universal ills, i.e., an Art of Living. "

These courses are taught by a man named S.N. Goenka, who learned the tradition from his mentor, Sayagyi U Ba Khin. There’s other traditions of the Vipassana meditation technique, and details can vary between different Buddhist sects, but largely, the fundamentals are the same. Goenkaji’s teaching is based in Theravada Buddhism, specifically the Satipatthana Sutta. However, for these courses it has largely had the religious aspects stripped from the teaching. There’s no mention of Theravada Buddhism in the course, and very little mention of Buddha at all. There are no statues on the course’s properties, and there’s no focus on one sect over another. You can be a member of any religion or none.

Modern day Buddhist sects as practiced in Asian countries certainly bolt on a lot of mysticism and rituals that appeal to that culture. Accept these rituals and rites if they are beneficial to you, or you may disregard them. It’s entirely up to you. They are not taught during the course, the focus is on 3 meditation techniques, with the primary one being Vipassana.

Arrival

Arrival day is day “0,” which starts sometime in the afternoon. You’re assigned to a room, eat dinner, and attend the introduction to the course. The rooms are basic, but adequate. Some centers have shared accommodations like a barracks, although most have small private rooms. I was given a room in a 2-room cabin. It was basic, clean and relatively comfortable.

There weren’t many westerners. I had expected it to be mostly tourists, but the course was total about 60 people, 85% of the course were Thai, and only 15% foreigners. A handful of the participants were Buddhist monks.

Noble Silence

Before going to your room however, you’re asked to turn in all of your valuables and distractions including mobile phones, books, laptops, etc. At Dhamma Kancana, you’re given a private locker and you retain the key so there’s no worry about security. The reason for this is Noble Silence. During the course you may have no contact with the outside world, nor communicate in any way with your fellow meditators. Complete silence of speech, body, and mind.

Your bags are not searched to determine if you turned everything in, this is really only for your benefit so you can focus on what you came to do. If you’re communicating, or otherwise stimulating your mind, you will lose your ability to concentrate. After a conversation, and you go into the meditation hall, your mind will still be buzzing with thoughts. It’s very difficult to concentrate and learn if you’re busy thinking about other things.

For me, Noble Silence wasn’t particularly difficult as I’m somewhat of a loner anyway, but the lack of being connected to the world, while jarring for the first couple days, I believe was the single most important factor for me being successful during these 10 days.

Precepts

As part of the course, you’re required to accept 5 precepts for the entire duration of your stay:

1. to abstain from killing any living thing
2. to abstain from taking that which is not given freely
3. to abstain from all sexual activity
4. to abstain from telling lies
5. to abstain from all intoxicants

The first precept, in addition to meaning you shouldn’t be killing the various bugs you might encounter in the jungle, also translates into a vegan diet; there are no animal products served during the course. Actually, Dhamma Kancana where I took the course is quite renowned for it’s food. It is some of the best vegan food I’ve had in my entire life, and I’ve definitely taken some ideas home with me. I was already vegetarian so I didn’t need much convincing.

As part of Vipassana practice, it’s recommended to continue these precepts after after leaving the course, however it’s not required for the practice, and some may be modified. There’s no magic in following these, and there’s no Flying Spaghetti Monster in the sky keeping track.

Goenkaji suggests that being vegetarian is good practice for developing mettā (compassion), however, Theravada Buddhists, as is practiced in Thailand for example, do generally eat quite a lot of meat. Other sects of Buddhism are quite strict about it. However, every Buddhist will readily admit that being vegetarian is the better option. As I’m fortunate enough to live in a wealthy country with many resources, I have a choice in what to eat, and I can’t see taste preference as being a justification to cause the suffering of another living thing.

Post-course, precept #3 is normally taken by lay Buddhists as avoiding “misconduct”, not celibacy. This is also not the Judeo-Christian view of “misconduct”. Sexual misconduct in the context of Buddhism is any sexual practice that leads to suffering. Consensual sex between adults of same or opposite genders is allowed.

For old students in the course, there are some additional precepts such as no “high and luxurious beds” and no eating after noon.

Schedule

The schedule is intense, beginning at 4 am and ending at 9 pm every day. All said and done, you’re meditating for around 10 hours per day. There’s a set schedule, with about half of the meditation sessions being allowed to be in your room if you wish. The room is more comfortable as you can sit on the bed with your legs hanging off, instead of having to hold an uncomfortable position in the hall. There is free time during the day to walk mindfully around the facility, do laundry, clean your room, etc.

Breakfast and lunch are substantial meals, but dinner is only afternoon tea: tea with fruit. They also had bread and jelly as well during dinner, and occasionally peanut butter. The days with peanut butter were nice as I could make a PB&J. The amount food is more than sufficient, you’re not doing a lot of physical activity and should eat less than you think.

The day ends in the late evening with a 1-2 hour video lecture by S.N. Goenka, followed by a 30-45 minute practice session of any new techniques, and an optional time period to ask questions to the teacher in private. Communication with the staff is allowed, it’s not breaking Noble Silence, but it should be infrequent.

The video lectures are some of the most interesting and engaging things I’ve ever listened to. Goenkaji is a prolific and interesting speaker. If you’re taking the course and speak English well, opt to view the videos in Goenkaji’s excellent English. I suspect his charisma is lost in the translations.

On day 10, you are free from your obligation of Noble Silence by mid-morning, and are encouraged to talk again. It was actually quite odd, I had spoken a couple of times to the teacher during the week, but when I went to say something at normal volume, my throat cracked and I couldn’t talk!

Techniques

Beginning on the evening of day 0, and continuing on days 1, 2, and 3, you are taught a “mindfulness of breathing” technique known as anapanasati. It is as simple as it sounds: you sit and be aware of your breathing. You don’t control it, but simply remain aware of the breath, the feeling as it enters and leaves your nose, which nostrils it is in, and the feeling on the upper lip. These first 3 days are useful to build your mind’s concentration.

If you’ve ever sat for meditation before, you know that it’s quite difficult to maintain concentration. The mind wanders every few seconds! Part of the exercise of the first 3 days of anapanasati is to build your ability to stay equanimous when this happens: you simply acknowledge the thought without feeling upset (building compassion for yourself while developing concentration), and you return to breathing. You’ll notice as the days continue, the mind begins to wander less and less.

On the evening of the 3rd day, you are introduced to vipassana. One of the core teachings of Buddhism is that everything in the universe is impermanent (the concept of anicca). Because everything in the universe is constantly arising and passing away, there’s no sense in being attached to it. Vipassana teaches you to avoid cravings and aversions for anything that happens in life. Craving and aversion lead to attachment, which always leads to suffering. Vipassana is a meditation technique that intuitively teaches you this lesson; it changes the habit pattern of the mind by giving you direct experience of anicca.

So, what exactly is vipassana? It has to do with observing bodily sensations. You’ll spend the next 7 days observing your body by examining piece by piece, and acknowledging but not reacting to the various sensations you experience. Sometimes they are pleasant, sometimes they are not, but they are always impermanent – they will always pass away.

It’s a simple but effective technique. You are changing your habit patterns in how you react – you develop equanimity, mindfulness, and compassion for yourself using this technique. It’s hard to explain, which is why you should take the 10 day course to learn it in detail!

The final meditation technique you learn is mettā bhavanawhich is taught in only a few hours on the last day. It is essentially what you should do at the end of every vipassana meditation session for a few minutes: it’s compassion meditation. Basically, you’re contemplating and “sending” (if you believe such things are possible) your compassion to specific people or people in general, and as well as yourself. It’s a way of wishing good feelings for everyone.

Results

I found the course beneficial. The technique is extremely helpful in becoming a more balanced person, and becoming more mindful of what’s going on in day-to-day life. I’ve definitely noticed changes in how I react to things that happened.

One interesting side effect in the days following the course was the incredible expansion of my attention span. Google is making us stupid, and blogs are destroying our attention spans. In the days that followed, I digested a number of pieces of long form writing, including three whole books – and the books were consumed mindfully, often pausing and spending time to think about what was being said. Vipassana is definitely a cure for at least this modern ill!

I haven’t maintained the practice perfectly, but seem to be able to meditate almost every other day. However, I was on vacation and in different places so it was hard to keep up. I hope to try to do the recommended 1 hour in the morning and evening now that I’m back home.

I won’t lie, the course was challenging. On day 0, as I was laying in bed without any of my electronic crutches like my iPhone, I was looking at out the prospect of 10 whole days dealing with this. It was frightening. From the outside 10 days seems like nothing; from the inside it seems like everything. Days are big, big things when you’re silent, alone, and disconnected.

For the first few days, I was counting meals because they seemed like much shorter time periods to adjust to! At the end of day 1, I kept saying “Ok, look, you’ve already had 4 meals, and you’ve hardly been here any time at all. You’ve only got 27 more, and they’ll go by faster than you think.” I worried about what was going on in the outside world, concerned even I hadn’t told my tax accountant I would be offline in case the Finance Ministry had questions about my tax return. The thoughts faded as the days went on, 27 meals became 18, 18 became 15, and eventually I stopped counting and became content with the disconnection from the world. It wasn’t without it’s difficulties, though. One day in particular was hard enough I contemplated quitting. Everyone going through this course will face some of their own demons, and will have a moment like I did. It could come on day 3, or on day 9, but it will happen.

The day that we learned the Vipassana technique, I could not get my mind under control. I was fidgeting non-stop, and it was everything I could do just not to walk out the gate to the center and go home. I was angry: angry at the teacher for making me sit so uncomfortably, angry at not being able to concentrate, and even angry at myself for feeling angry. At the end of that 2-hour session, I went back to my room and laid in bed, quite upset at how I reacted, how I hadn’t paid enough attention to the teaching, etc.

I began to think how far too often I’ve made knee jerk reactions in life to uncomfortable circumstances. I did anapanasati, and calmed myself down. I returned for the next session and made it through the remaining 7 days without significant problems

It’s challenging, you’ll be pushing yourself to the limit of what you think you can do. However, if you persist, you’re bound to be successful.

Bhavatu Sabba Maṅgalaṃ (May All Beings Be Happy)

With only one full day left of vacation visiting my folks in Albuquerque, my brother and I were brainstorming things I hadn’t done before in the “Land of Enchantment.” He mentioned the Carlsbad Caverns, but we crossed it off the list because of the long drive…until I remembered I had seen flights to Carlsbad on the departures board at Albuquerque.

After some research, I learned that these flights were offered under the Essential Air Service program which provides federally-subsidized air service to remote cities that would otherwise be isolated. One of the largest companies in this space, Pacific Wings, has several “brands” like Georgia Skies and New Mexico Airlines that operate these contracted flights.

Booking

The New Mexico Airlines web site, like all of the Pacific Wing airlines, is a rebranded version of the parent company’s site. The web site itself was really easy to use, and return tickets to to Carlsbad were $187 each.

I booked the first flight out to Carslbad, LW02, departing at 7:45, and returning on LW86 at 16:45. Plenty of time for a day trip to the caverns.

I paid for the flights with my credit card, and received a confirmation e-mail a few minutes later with a true blue PNR. I wasn’t expecting that they had a real travel system on the backend. It turns out they really are a fully integrated airline with Sabre and Amadeus, complete with baggage interlining agreements.

I was curious to see if this airline was visible in ExpertFlyer. It was – it showed Y7 on the outbound flight, with the aircraft’s capacity at 9. It looked possible that we the only people booked on the outbound…

Unfortunately, Carlsbad is a one horse town – or rather, a one taxi town, that does not go to the caverns. There is, however, an Enterprise car rental in the city who will pick you up and drop you off at the airport.

Check-In

The instructions on the web site were clear that you must be at the gate an hour before departure, and I was a little worried I’d have a hard time locating the check-in counter. We arrived early, but my worries were unfounded because there’s a big sign we spotted immediately after entering the airport that pointed the direction to the New Mexico Airlines check-in desk.

No one was staffing the desk, however there was a phone and a sign instructing passengers to pick it up. I was routed immediately to the Pacific Wings central call center, and the friendly rep completed our “pre-check-in” and gave us directions to the gate (100 meters away). She told us the pilots would be at the gate shortly to check us in. Interestingly, there is no TSA security check for these flights, you can walk directly to the boarding gate.

The gate is next to the NMA office, and a bunch of the staff were buzzing around. I had a friendly chat with one them. Today was a big day for them, they were breaking-in a new (to them) plane. The livery had just been done the day before, and they were flying it up to Los Alamos for a dog and pony show. That brings New Mexico Airlines total fleet to 3 Cessna 208B’s. I got a chance to see the new plane, N307PW:

I looked up pictures of the Cessna 208B’s before I left, and saw some from Georgia Skies, but none from New Mexico Airlines. The Georgia Skies Cessna’s didn’t seem to have much beyond a default boring plane scheme, so I was pleasantly surprised to see such a good looking livery on these NMA planes.

At exactly an hour before our scheduled departure, a pilot showed up at the check-in counter to take our weights and get our information. It was here I began to worry: our pilot looked really young – like he had skipped high school classes to fly this morning. He was also fidgety and nervous, and could have also used a lesson or two on interacting with customers. Thankfully he wasn’t actually the Captain, just the co-pilot who was still learning…

During check-in, he confirmed we were the only passengers, and then angrily ranted about how the EAS program is a waste of his tax dollars, and he’s often flying these routes with empty planes because of their contract with the Federal government. I thought it was strange, considering that’s where his pay check comes from. Also, if you lived in Carlsbad about four hours from the nearest big airport, you’d probably disagree about the worth of the EAS program.

Boarding

About 30 minutes before our departure time, the Captain showed up and it was a night-and-day difference from our co-pilot. He was older, and had that experienced aviator vibe that can make the most nervous passenger feel at ease. At 15 minutes before departure, the co-pilot made the announcement to board and we exited the terminal through the glass doors directly onto the tarmac.

As we were boarding the plane, the Captain asked us to sit in the last two seats, I imagine to keep the weight/balance in check.

Cabin

The seats were more comfortable than I imagined, with plenty of leg room, although the lack of a head rest made sleeping precarious. The seat belt was also over-the-shoulder, and shorter than you would expect. A taller person would be pretty uncomfortable lashed to the seat.

The plane is equipped with a single set of indicators for seat belts and no smoking. I do wonder, though, when it would really ever be safe on such a small plane to take off your seatbelt.

Each seat also had a reading light and air vent:

And an oxygen mask in the seat pocket, that would be connected directly to the air vent:

Safety Information Cards:

Each seat was also equipped with a copy of open letters between Pacific Wings and the City of Carlsbad arguing about the EAS contracts. Small town politics at work.

Flight

After we were on board, the pilots completed their take-off checklist, and we began taxiing to the runway. During the taxi, the co-pilot gave a rather dispassionate and fast security briefing that was a bit hard to understand. We were in the air quickly after that, and take off was smooth.

This was my first time on a single-engine prop plane, and there was not a single bit of turbulence on the way down to Carlsbad.

Directly around Albuquerque, I was treated to some scenic views through the Cessna’s big, big windows:

But once we left the Albuquerque, the scenery was not quite as exciting, desert as far as the eye can see:

We were in the Carlsbad area about an hour after take off, and things started to look a bit greener:

Landing was as smooth as the take-off.

When we arrived at CNM, the Captain told us to be back by 16:15 for our return flight. The walk to the terminal building was about 50 meters, and at the door, the Enterprise representative was waiting. Luckily, we didn’t even need to go into the city: Enterprise had brought the car to us at the airport, and we completed the paperwork in the terminal. I wish the rental car would always pick me up at the door to the plane! It was very nice of them, that saved us about an hour of time!

Carlsbad Caverns

The Caverns are about a 30 minute drive from the airport:

We got to the caverns about 9:50a.m., too late to join the 10 a.m. tour, so we did the self-guided tour with audio guides. The Caverns are incredible!

All in all, it took us about 3 hours to make it through the caverns. After eating lunch at the restaurant, we still had about 2 hours to kill before our flight. After some Googling, I found that the Living Desert and Zoo State Park was near by, so we headed over to take a walk around:

Everything in the park is native to the Chihuahua desert, except for the green house at the end which contains a variety of Cacti from around the world. The most impressive of which was around 15 feet (4.5 meters) tall!

Return Flight

We left the gardens around 15:30 and were back at the airport by 16:00.

The check-in counter is the same process as Albuquerque:

We waited in the small seating area until the pilots came again to check everyone in:

By 16:40, we were again on the aircraft, although this time it was relatively full with 7 passengers. The pilots told us in advance that it would be much bumpier than on the way down, due to high winds. It was indeed pretty turbulent, although I’ve had much worse on bigger jets. The Cessna took it like a champ, and the pilots were doing a good job of trying to mitigate it as much as possible. We spent most of the flight at a pretty low altitude to avoid the bad air.

The flight back was about 20 minutes longer due to the winds, but we landed in Albuquerque right on time around 18:05. We were home in time for dinner. The total trip time was around 10 hours. Driving alone would have taken that long round-trip.

I was really happy with my New Mexico Airlines experience, and will definitely look for other opportunities to fly on these EAS routes. If you’re lucky, you can get a private plane experience for a fraction of the cost.