The Internet Of Things is as much about Big Data as it is about connected devices. This series will take a hands-on approach to getting data from a device, pushing it to the cloud, and finally representing it in a meaningful way.
I have been playing around with microcontrollers for almost two decades now but this hobby has been mostly idle for the past 5 years, replaced by woodworking projects. Since my workshop is in the garage, and winters are cold here in Chicago, I decided to resurrect the microcontrollers and see if I can’t merge the two interests, maybe even finish my greenhouse I started nearly 8 years ago. I believe a greenhouse is a perfect hotbed for a microcontroller playground.
So I decided to do a series of posts about some of the things I have recently learned, specifically about the new generation of MCUs (MicroController Unit) that include features such as Wifi, Linux, and full package management. My winter goal is to build a windowsill greenhouse so I can experiment with monitoring and controls which will motivate me to complete the full-scale version in the spring.
Primer on Microcontrollers & IOT
Any discussion on the Microcontrollers would not be complete without starting with Microchip’s PIC. MCUs have been around for a very long time and exist in everything from your kitchen appliances to industrial applications. The first ones had EPROM memory which could only be programed once. This is ok if you are making toaster and the code never changes but it wasn’t very useful to hobbyist like us. When Microchip added a Flash memory they opened up a new market for folks who wanted to tinker. Initially you had to buy a programmer for about $300, (I have one for sale if you want it) but eventually they added support for in-line programming straight from a serial port on a dev board. This was a huge breakthrough. Although the re-programmable PICs were only a few bucks you had to write your code in assembly. If you are a modern web developer try writing in assembly for a few days. It will make you bark like a dog.
In 2005 the second greatest export from Italy was unveiled to the world. It was an open source hardware project based on C and C++ instead of assembly. It was open source so anyone could make and sell Arduino clones, and the fact that it was based on C++ was huge since this is the foundation for most modern programming language. For example if you wanted to create an LCD on the PIC you had to bit bang every pixel, which is not scalable. But on the Arduino you just include the LCD library and define the dimensions (columns and rows) and do something like lcd.print(“Hello World!”) and the library would do the rest.
Arduinos caught on fast and another great feature they had were stacked-through header pins. This means you could put a board on top and pass the pins through to a second board if they were not used by the first. This concept was stolen from PC104 and PC104+ PC form factors which used a stackable ISA and PCI header pins respectively. Arduino calls these stackable daughter boards ‘shields’ and opened a whole new market for 3rd party producers.
The most popular shield was the ethernet Wiznet W5100. This was the primordial ooze of IOT. You could load ethernet, and client/server libraries and you could even make your Arduino call out to the Internet. Even though this was a breakthrough it was still a bit clunky and the client server code took most of the sketch space. Some folks even created MQTT libraries for the Arduino which allowed for much more efficient use of the sketch and asynchronous programming.
The next big player in this space was the Raspberry Pi and this controller was a game changer! If the Arduino was the primordial ooze, the Raspberry PI was the lungfish. This controller also learned lessons from the PC104 form factor. They implemented the System on a Chip (SoC) concept which meant you could run Linux. The market for PC104 was mostly industrial automation (I have a couple collecting dust) and the basic premise is a small processor that requires little if any external cooling and minimal power consumption. The Raspberry PI Foundation took this SoC and added ethernet, USB, HDMI, audio, and an SD slot in a 3.5 x 2.25″ form factor. They sold it with a version of Debian pre-installed, all for only $35. These peripherals would have cost in excess of $400 on a PC104 platform since they needed to be added individually via the stacked expansion.
In addition the PI include 8 (now 17) GPIO (General Purpose I/O) pins. GPIO defaults to digital but they can be configured to use PWM to simulate analog. This is typically good enough for most hobbyists. Although the IO pins were somewhat limiting compared to dedicated analog and digital ports, they still did the trick. The PI opened up an audience that was not limited to just C/C++ and its built-in networking and OS made it simpler to get started.
I have not powered up my Edison yet, but I have done a lot of research. The Edison Development kit is Arduino compatible, meaning that you can use the Arduino IDE to program it yet you still have access to the Linux side. It uses a library called mraa to map between the two processors just like the Yun (see below). Like the Raspberry, the Edison also uses GPIO pins rather than dedicated digital and analog and it appears to be the best of both worlds offering both Linux flexibility and Arduino stability.
I love this MCU! The Yun is Arduino’s response to the Raspberry PI and Edison. Like the Edsion it has two processors. The first is the classic Arduino processor based on the ATmega32u4 and you flash ‘sketches’ written in C++ to it just like you would with any other Arduino. However the really cool thing about the Yun is it has a second Linux processor using the Atheros AR9331 chipset running OpenWRT. OpenWRT is a small Linux distribution that is mostly used for Access Points and consumer routers. This hardware hosts the built-in Wifi and ethernet. The Yun has a Linux management interface called Luci that pretty much lets you do all the linux sys admin functions from a web console. Yeah, it is pretty slick!
The two processors talk to each other via a Bridge Library that allows for two-way communications between the two processors. In fact there is a sketch called Bridge.ino that wraps digital and analog pin read/writes/pinmodes into a REST call hosted by the Linux side’s web server. This is very different from the approach that the Raspberry PI and Edison take. They use GPIO pins and allow the Linux software to access them via an included library.
I like the way the Yun does it because I can choose to put the logic inside the Arduino sketch or put it in some code running on the Linux platform. I will refer to this code running on the Linux platform as an Agent. Another cool feature of the Bridge is that you can make your Arduino Sketch execute commands on the Linux side.
I want to do several variations, or ”recipes” of the same thing: getting sensor data from an MCU to a cloud system to analyze and act upon the data. I want to compare different MCU and different presentation layers so the reader can make informed decisions based on the architecture they are most comfortable with. Below is a short list of the technologies I am planning to use.
IOT Ingredients in the Series
- Arduino Yun
- Raspberry Pi
- Intel Edison
Data storage and presentation layer
- Treasure Data (Hadoop)
- Temperature: DS1820B
- Moisture: Homemade using two galvanize nails
- Light: photo resistor
- A Web server that can host d3js html pages.
- A route engine to create some custom routes: say, if I only want moisture or if I want to aggregate daily averages.
- Maybe some authentication with Oauth.
Teaser: Recipe 1
YUN->Node Agent->Mongolab->d3.js running on Mean.io hosted on Heroku
Before we get started I wanted to give you a brief teaser for one of the first recipes I will cover in the next post. The image below shows an Arduino Yun with the above-mentioned temperature, light and moisture sensors attached.
Actually there are two temperature sensors, one light sensor and one moisture sensor. The Arduino is mounted to a project board that has an area to the left for a small breadboard. I usually find this small breadboard is sufficient.
Photoresistor: Temperature sensor:
The second temperature sensor is connected via the black cable next to it and is against the window. Since the DS1820B uses the 1-wire protocol you can connect as many as you want using a single digital pin. You can’t see the moisture sensor since it is burried in the plant. But I will go into more detail on this circuit in the next post.
ATmega32u4 – Arduino Sketch:
On the Arduino I am running a modified version of the bridge library. By default the bridge library allows you to read or write any analog or digital pins via a REST call. I need to add a route that would read the temperature from the 1-wire bus. It should be noted that the DS1820B is a digital temperature sensor with a 64bit address to be used with the Dallas Temperature library. When you call
you will get a discrete value in Celsius. As an alternative you could use an analog temperature sensor like the [LM335)[http://www.ti.com/product/lm335] connected to an analog pin. When you read the pin you will get a 10bit (0-1024) value back based on the supply voltage of 5V which corresponds to a temperature that you must calibrate. I will supply a link to my modified bridge in the next post.
Atheros – Linux Code:
I like Linux! I also like NodeJS. So I wanted to write my agent in Node. Unfortunately the OpenWRT takes 9 of the 16MB of the onboard flash on the processor so it does not have room for Node. Yeah, I could have done it in Python or even Bash but I wanted Node. So I followed this tutorial which explains how to add an SD card and mount the rootfs from the SD slot. I now have all kinds of room.
I wrote a simple node agent that runs every 5 minutes via a cronjob that uses request.js to get the sensor readings via a REST call to localhost. It then posts it to Mongolabs via their REST api. I also used moment.js to get both the epoch time and the UTC time and add it to the data. (More on that in the next post.)
Tip: My vi skills are strong, however vi can’t compare to all the plugins provided by modern editors like Atom or Sublime. But the problem is the node code lives on the Yun so I need to mount the drive. What to do? No problem! Install open-sftp-server on the Yun via opkg and mount the drive on your Mac with the ssh file system. My yun is called kyleyun.local, so on my Mac I simply type: sshfs firstname.lastname@example.org:/ ~/mnt/yun then I can run Atom from this mounted drive.
storage – mongolab.com
Sign up for a free mongolab account and create a database and a collection. My database is called yun and my collection is sensors. When you get an account you can also generate an api key that allows you to POST/PUT/GET/DELETE any data to that collection. So posting to Mongo with my agent was trivial.
Application -Heroku + Mean.io + D3js
So I get data into my db every 5 minutes with little effort. Now I have the hard part. Here is what I need:
So I used Mean.io, a great Mongo, Express, Angular, Node framework. I am going to save the details on this one for the next post but here is a sample graph of my daily moisture averages.
You can see an index of all my graphs here on Heroku but I must warn you they are very basic. D3.js can take a long time to master and I’ve just started to scratch the surface. Also, this is hosted on the free version of Heroku which goes idle every two hours so you may need to hit refresh if you get a blank screen.
This post serves as an introduction to this series. I talked a little bit about the history of Microcontrollers and about the next generation of natively connected devices. The follow-up articles will dive into deeper detail on each of these projects. I will always follow the same premise: getting sensor data into a Cloud enviroment but I will swap out various components on each post and discuss the pros and cons of using each. At times this may seem like a Rube Goldberg machine but it is intended to show different approaches. These projects are built on the micro service principal that uses many small bits of code that can stand alone or be swapped. If you have any suggestions for future experiments I would love to hear your thoughts. You can email me at kyle dot bowerman at appirio dot com (email@example.com).