My “smart house” started out a few years ago as a couple of Hue light bulbs in my apartment. Now that we’ve bought a house, I started acquiring a few more smart things here and there, and over the last couple of months I finally decided to integrate them all together using an open source project called Home Assistant.

To be honest, I didn’t start out with a plan, and I wish I had. However, with few exceptions all the things I’ve bought play nicely together. Home Assistant has allowed me to develop a number of interesting things.

When no one is at home, the house puts itself in away mode: lights and televisions get turned off, my thermostat temperatures set to Eco mode, and all of the doors get locked. Presence detection is done through the Unifi module, which looks for devices on our main and Guest WLAN’s. Solar production is tracked, and in case of a grid failure, power-hungry devices like the space heater in our sunroom, or the A/C are turned off or adjusted.

Home Assistant is also fully integrated with Amazon Alexa through the HA Cloud project. I’m able to control all my devices through HA instead of using each platform’s integration. Custom skills even do things like report on my solar system, or answer questions about if someone is home.

I also use HADashboard and an old iPad mounted to the kitchen wall:

Home Network

Goal number one when we moved into the house was getting it wired for CAT6, and a small equipment rack installed in the basement. I switched over from DD-WRT to Unifi last year, and now that I’m in the house I’ve bought a few more Unifi things. My network consists of:

• UniFi security gateway
• 2x 8 port 60W POE Switch
• 1x 8 port switch
• 2x UniFi AP-AC-Pro

The main challenge was getting the CAT6 to the various rooms, and learning how to punch down a keystone jack. I ended up buying a spool of CAT6 on Amazon along with keystone jacks, wallplates, a cable tester, etc. It was slow-going at first, but I managed to finagle network drops to all the rooms I wanted: living room, office, kitchen for the access point, and the basement family room. Each room has several drops going to it, as I never really want to have to do this again! Although eventually I plan to add some POE cameras, so I may have to run some network drops outside.

The network is split into 3 VLAN’s, with limited connectivity between each:

• Main network
• Guest network
• Internet of Shit things

Lighting

My first “smart house” devices were Hue light bulbs, which work very well and have a wonderfully open API and developer-friendly attitude. I still have quite a few Hue devices, and plan to add a few more like the Hue Go. For wall switches however, now that I actually owned the property I could use a little bit more full featured smart switches instead. All my reading lead me to narrow down my options to Z-Wave switches from GE, Lutron Caseta and Insteon. Insteon has a large ecosystem of smart things, and came highly reccomended from a friend of mine. But Caseta devices were easily available locally, and they’re pretty, especially with the Claro wall plates.

I had heard bad things about Z-wave range and reliability, although I do actually own a few non-lighting Z-wave devices now. The downside to the others is that both Lutron Caseta and Insteon are proprietary protocols, with proprietary hubs. Lutron needs the Pro hub if you want an open telnet port for reliable integration with third party tooling. The consumer hub has been reverse engineered from the Android app, and there’s a Home Assistant component but it breaks every so often.

Z-wave seems to be the way to go if you want a more open protocol, and in retrospect now that I own a few Z-wave devices, I would probably have used it for my lights too.

But, I am happy with Lutron. It works well with Home Assistant, and their customer service is top notch.

Nest Ecosystem

Through my utility provider, I bought a Nest thermostat at a substantial discount. It was my first Nest product, and despite being owned by Google, has a pretty open API. I later bought Nest Protect fire alarms: they looked nice, and the nightlight feature was really handy to light up the hallway to the bathroom at 3 a.m. I’m very happy with these products. They integrate well with Home Assistant.

My last Nest product, however, was a different story.

Locks

Being happy with the other Nest products, I decided on buying a Nest x Yale lock. I made some assumptions that were wrong, based on my experience with other Nest devices and the “Works with Nest” logo on the box.

The lock has no API. It has no Amazon integration. It does, however, have integration with Google Home. I believe this is intentional on Google’s part to close down new Nest products from third party services, and suck people into Google Home. I sold my lock on eBay.

Now on the hunt for new locks, I came to the conclusion I might as well bite the bullet and get into Z-wave. I’m glad I did. I bought 2 used Yale 210’s on eBay, and a Schalge Camelot for the main door. They didn’t work seemlessly, but I ended up needing to write a patch to fix some quirks.

Solar

A big consideration for me when I bought the house was having a good potential for solar. My house isn’t in a perfect position, as it faces southeast/northwest, but I was able to get panels on both sides meaning in the end I fit a bigger system than I could have otherwise. I have 30x 295W panels, for a total of 8.85kW, and an underclocked SolarEdge 7.6kW inverter. In the first month of operation, I produced over 1100kWh - more than double our usage. The net metering credits from that production should help in the winter when there’s less sun.

In addition to the solar, I have a Powerwall 2. The Powerwall operates in backup mode, and will take over in case of a grid failure, powering the house by battery and available solar. Home Assistant sends out notifications, and enacts power conservation measures like adjusting the thermostat, and turning off power-hungry devices.

I’ve also integrated information about my solar from the SolarEdge API using Home Assistant’s REST sensors, which displays nicely on HADashboard:

A few years ago, I finished my B.S. in Computer Science through UMUC, online and while traveling as a Consultant for Red Hat. When I moved back to the US, I knew I wanted to use the remainder of my GI Bill entitlement on a Master’s degree. These days, there’s a lot of options for online programs - for example, Georgia Tech’s online MSCS program. My coworker did a nice write up of his experience so far.

I decided to opt for a local program, and narrowed down my choices between Tuft’s part-time M.S. in Computer Science, and Harvard Extension’s awkwardly named Master of Liberal Arts in extension studies, field: Software Engineering. There’s a difference between computer science and software engineering, of course, but the Harvard ALM was flexible enough that I could include the theoretical stuff I wanted and it was easier to get to Harvard on public transportation. The degrees from the extension school seem to be awkwardly named to differentiate the Extension school from the other more traditional schools at Harvard. Still, I don’t get why it’s a Master of Liberal Arts, that’s not really reflective of the coursework. There’s been some effort to change that, although I doubt I’ll ever see it as I’m nearly done.

I’ve mostly taken one course at a time, but this year my awesome employer graciously let me take a leave of absence to study full time to wrap things up quickly as my GI Bill benefits expire soon.

Update: I’ve since returned from my LoA, and finished all but my last course (the Capstone). I’ve updated this blog post with reviews of everything I’ve taken so far, and some conclusions I’ve made so far about the ALM.

Courses

• CSCI E-97: Software Design Patterns
• CSCI E-93: Computer Architecture
• CSCI E-95: Compiler Design
• CSCI E-92: Operating Systems
• CSCI E-28: Unix Programming
• STAT E-100: Intro to Statistics
• PHYS S-123: Laboratory Electronics: Analog and Digital Circuit Design
• CSCI E-55: Java, Hadoop, Lambda Expressions, and Streams
• CSCI E-15: Dynamic Web Design
• CSCI E-134: Networks
• CSCI E-48: Secure Mobile Computing
• CSCI E-599: Capstone

CSCI E-97: Software Design Patterns

This class was a deep dive into software design patterns. It taught most of the key patterns from the “Gang of Four.” It was also a heavy deep dive into UML.

I thought the class was useful, but the material was dry. It is a required part of the ALM.

CSCI E-93: Computer Architecture

This and the following two classes (CSCI E-95, and E-92) were my best experience at Harvard so far. I used this class as my “theoretical foundations” requirement for the 3 admissions courses.

In this class, I built a computer processor from scratch. I designed an instruction set, wrote an assembler, an emulator, and then finally implemented the processor using VHDL that ran on a physical piece of hardware (an FPGA development board from Altera). My final project video is on YouTube.

This and the other 2 classes are designed such that if you work through each of the problem sets, you’ll get a working thing in the end. Out of all 3 classes, this one had more freedom as you could really design any kind of processor you wanted. Some people tried more adventurous things like stack machines or getting pipelining working, but this was my first experience at this layer so I ended up doing a 16-bit, mostly MIPS-like architecture. My final project’s special feature was an LFSR. Many students opt for interrupts or hardware multipliers as theirs.

If you end up taking this, go to section. It’s extremely useful and a lot of implementation suggestions are given.

CSCI E-95: Compiler Design

I wrote a compiler for a large subset of C using C along with classic compiler tools (Flex and Bison). Whereas many other universities teach classes where you only learn theory, or only implement a “simple” language - this class stood out in that you literally write a compiler for C and have to understand all of its quirks.

It’s essentially C89 minus structs, unions, and function pointers. The final project is implementing an optimization stage in the compiler, mostly adding simpler peephole optimizations on basic blocks. Many students also work on more advanced register allocation strategies like graph coloring.

My final project video is here.

It’s worth noting, I didn’t know even know C when I started the class, but I knew it pretty well by the end.

CSCI E-92: Operating Systems

In this class, we learned the important operating system concepts, and then implemented an OS on a Freescale K70 Tower. You start out writing a small shell, your own implementation of malloc, various system calls in the OS, and towards the end of the class everything comes together when you write own scheduler and get multitasking working.

My final project was a rather complete POSIX-like permissions system, and interrupt-based Semaphores. I also got multiple serial ports working on the device, which made the demos a little more interesting.

My final project video is here.

CSCI E-28: Unix Programming

This class dives into the details of how POSIX systems programming works (and more specifically, the class touches on a lot of Linux-specific things). I took this concurrently with CSCI E-92, which it was a nice complement for. I’d reccomend taking this first, or also concurrently, to see how real operating systems design their system calls.

During this class, I wrote a shell (a bit more complex than the one I wrote for CSCI E-92), a pong game using curses, as well as a multithreaded web server for my final project. I wouldn’t say this is a particularly demanding course if you’re already familiar with Unix-like operating systems and know C. Most of the problem sets come with significant starter code.

The class is taught by the author of Understanding UNIX/LINUX Programming.

STAT E-100: Intro to Statistics

This class was underwhelming. I was hoping for a deeper dive into R, but it was much more using R as a REPL with 99% of the R code given to us. It provided a good introduction to statistical concepts, but was rather shallow in the depth of the topics that were covered.

I also took this as an online-only class, as it was the only stat class that’d fit into my schedule. The video lectures were really great, however the problem sets were not very challenging and mostly multiple choice. I expected it to be a little harder, and was disappointed I blew one of my elective slots on this class considering I could’ve got this out of Khan Academy on my own.

There’s a few other statistics classes at Harvard Extension (100, 102, 110, etc), offered by a number of different instructors. Perhaps some are better than others.

PHYS S-123: Laboratory Electronics: Analog and Digital Circuit Design

This was an 8-credit (2 course) summer program that ran Monday through Thursday, 9 to 1pm (officially) over seven weeks. On days with labs, the time was more realistically 2:30 or 3pm. Add on homework and study time, I was getting home in the evening nearly every day.

Typically, this is taught as two separate classes: a semester on analog electronics and a semester on digital electronics. This summer school version is intense - covering this amount of material in 7 weeks is daunting, and for the summer course some of the more interesting things are removed. Instead of the “big board” path where you build up your own computer on breadboards, we worked with a SiLabs microcontroller that had most everything built-in.

Still, it was a great experience and I’m glad I took this class. The analog section starts off covering voltage, current, and resistance and building passive circuits. It moves on to transistors, both BJT and MOSFETs, and you go on to build an op amp from discrete parts to understand what’s inside. It goes on to cover op amps in detail covering usages of positive and negative feedback, and on the final day of the analog part of the course, we designed and built a group project that transmitted and received an audio signal using infrared light.

The digital part of the class starts off with boolean logic, HDLs, and logic gates. CSCI E-93 covered a lot of this, but the electronics were abstracted away from us in VHDL. In this course, you look at what’s actually inside both TTL and CMOS logic gates, build analog-to-digital or digital-to-analog converters from parts, design and build state machines using flip flops, etc. The final week of the class is working with the SiLabs microcontroller, and writing assembly programs for it.

The text book we used is The Art of Electronics, along with the accompanying student lab manual.

CSCI E-55: Java, Hadoop, Lambda Expressions, and Streams

This met the ‘cloud’ requirement in the ALM program – because of the short section on Hadoop. It was the only available class that met the cloud requirement, so I had to take it. It would be a useful class if you did not know Java, but were already an experienced developer in something else. I did know Java so it was largely review. The overview of Java 8 features, and Hadoop were great, but that only made up about the last 1/4 of the course.

The instructor is great, and provides a history of his experience as a software developer over several decades. That was my favorite part of the class.

CSCI E-15: Dynamic Web Design

Another requirement for the ALM is ‘web design.’ The course was very well organized, and helpful if you knew HTML/CSS and did not have any background with a MVC framework. The course teaches Laravel, PHP, and HTML and students get a handle on git and work a bit with the LAMP stack.

For me, a waste of time and I wish ‘web design’ was not required, and instead given another elective spot.

CSCI E-134: Networks

This course I had intended to fulfill my ‘data communications’ requirement for the ALM. It was an exploration of all kinds of networks, such as social networks. It was marketed as an intersection of economics/computer science. It was run concurrently with the Harvard College version of the class.

I dropped it early on, as the advertised pre-reqs were not correct. One needs a deeper background than I had in math, including linear algebra. I’ve always regretted not having a better background in math. Maybe something I’ll improve after I graduate.

CSCI E-48: Secure Mobile Computing

Instead of E-134, I took this for the ‘data communications’ requirement.

The video lectures were largely reading the text slides word-for-word, unfortunately. Students do get some experience with packet sniffing WiFi networks with kismet and such, and some content related to 3G/LTE mobile networks. However, I didn’t like the course structure. Weekly requirements were forced discussion: watching a video and writing a summary post, and then replying to two other students’ posts.

The course staff was in general great, and I found them knowledgable and the labs OK, but I was hoping for more. Labs were interesting, but not very deep. Had I known CSCI E-134 wasn’t going to work out, I’d have taken one of the internet architecture courses on offer, but they did not fit with my schedule by the time I dropped it.

CSCI E-599: Capstone

Currently in progress for Spring 2018, will update when complete.

Conclusion

I have finished all but the Capstone, which I’m currently taking in Spring 2018.

There are many amazing courses at HES, but the 5 elective slots in the ALM program are not enough, especially when they’ve filled up the requirements with fluff like web design, “cloud”, and data communications where the classes only very loosely fit into those buckets.

I am a little bit disappointed by how many courses I felt like were wasting my time. My own professional goals do not align with how HES has structured the ALM. I’d rather have more elective slots, which I could’ve used to correct my weakness in math (MATH E-23A and others), or get more experience with research (CSCI E-191).

However, overall my experience at HES has been a good one. There are a number of classes that I think were high quality and well worth my time. I wrote a compiler, an operating system more or less from scratch, and designed my own instruction set and processor. I’ve built op amps on breadboards from transistors, and a combinational lock out of push buttons, LED’s, and flip flops. I’ve written a multithreaded web server, my own shell, and my own implementations of a ton of Unix utilities. All of this gave me a better foundation in how computers actually work, something I don’t think I had a good grasp on before I began this journey - even though I had been working as a developer and sysadmin for a number of years.

For 10 out of the last 15 years, I’ve run my own mail servers. I switched to Gmail when it came out, but I decided I didn’t want to have so much data in their hands. So I took it away from them, and I’ve hosted it again myself for the last few years. In the beginning, it was troublesome, I’d end up in spam folders everywhere. But these days I don’t have much issue with Yahoo or Hotmail/Outlook/MSN/whatever they are now - only Gmail is the outlier.

A few days ago, I attempted to e-mail a company regarding an online e-commerce order I had placed, from my personal address. My email was rejected by Google and improperly identified as bulk e-mail. It’s not an infrequent occurrence. Either my mail ends up in spam folders, or they outright reject it:

The e-mail headers google shows are fine - it passes both DKIM and SPF:

My mail server is not in any blacklist, and any kind of diagnostics you want are GREEN. I’m not an open relay, nor have I ever been.

What is my crime? I host a well-behaved mail server that’s been around for at least a few years, that implements industry standards like DKIM and SPF correctly. It has never sent a single spam, and barely sends mail at all - a couple hundred a year, maximum. And all of a personal nature.

I can only think this is intentional on Google’s part - they have a near monopoly; the vast majority of mail I send these days goes to Google - and if a small company is running their own mail server is too much of a hassle, then maybe they’d buy Google Apps. It’s bad, anti-competitive behavior on Google’s part. Shame on them if its true. I don’t know if it is, I can only guess, but they certainly have an incentive to make it difficult for the little guy.

I’m just a geek that likes running my own servers. My pleas to Google’s impersonal forms fall on deaf ears, and I’m getting tired of telling everyone I e-mail to check their spam folders.

What can I do except move to a hosted provider with a better reputation with Google?