Coding Chase - Projects

My 3D Hub

Thank You For Donating:

Ikeeki (linux-sunxi@irc.freenode.net)
Computer Troubleshooters Brasil

Sunday 12 July 2015

IoT Projects - ESP8266 [CTC-3D Printer Upgrade]

Based on the previous blog posts on MQTT, ESP8266-01 and NodeMCU, I decided as mentioned before to upgrade my newly acquired 3D printer, a CTC-3D dual extruder printer.

The printer is awesome and while it already operates remotely via Octoprint on Cubietruck (see previous posts), I did find it useful to add a cooling system to it as well as some lighting inside the chassis, both to be remotely controlled as well.

This post is all about the lua scripts loaded to the ESP8266-01 wifi module, in order to achieve this.

First off, I had several issues with memory usage while using a stock nodeMCU firmware, meaning, a build containing all the available modules.

Trimming the fat from the firmware proved quite valuable in memory terms (check previous posts).

Then, moving on to the lua scripts, I managed to devise a basic management system where I can now update the wifi module configurations for WiFi and MQTT broker over the air, using the MQTT messaging system.

config_wifi.lua:

wifi_ssid='wifiHotspot'
wifi_pass='w1f1P4ss'

config_mqtt.lua:

mqtt_host='192.168.1.2'
mqtt_port=1883
mqtt_topic='topic-'..node.chipid() 
 
bootup.lua:
 
function bootup()
    wifi.setmode(wifi.STATION)
    wifi.sta.config(wifi_ssid,wifi_pass)
end




mqtt_connect.lua:


function mqttActivate()
            m:connect(mqtt_host, mqtt_port, 0, function(conn)
                print("connected to mqtt broker!")
                m:subscribe("/"..mqtt_topic,0, function(conn)
                 tmr.stop(3)
                 tmr.stop(2)
                end)
            end)
end

function mqttConnect()
  m = mqtt.Client(wifi.sta.getmac(), 120, "user", "password")
  m:lwt("/lwt", wifi.sta.getmac(), 0, 0)

  m:on("offline", function(con)
       tmr.alarm(2, 10000, 1, function()
        print ("reconnecting to mqtt broker...")
        mqttActivate()
       end)
  end)

  tmr.alarm(3, 10000, 1, function()
        print("connecting to mqtt broker...")
   mqttActivate()
  end)

  mqttParse()

end

mqtt_parse.lua:


function mqttParse()
  m:on("message", function(conn, topic, data)
    if data ~= nil then
      local i=1
      local offset=0
      while string.find(data,";") do

       field = string.sub(data, offset, string.find(data,":")-1)
       dataArray[field] = string.sub(data, string.find(data,":")+1, string.find(data,";")-1)
       data = string.sub(data, string.find(data, ";") + 1, string.len(data))

       i = i + 1
      tmr.wdclr()
      end

      if dataArray['type'] == 'config-update' then 
       confUpdate()
      elseif dataArray['type'] == 'gpio-control' then
       gpioControl()
      end
    end
  end)
end
 
config_update.lua:

function confUpdate()
 print('changing configuration for module: '..dataArray['setting'])

 if dataArray['setting'] == 'wifi' then
  file.open("config_wifi.lua", "w+")
  file.writeline('wifi_ssid=\''..dataArray['ssid']..'\'')
  file.writeline('wifi_pass=\''..dataArray['pass']..'\'')
  file.close()
 elseif dataArray['setting'] == 'mqtt' then
  file.open("config_mqtt.lua", "w+")
  file.writeline('mqtt_host=\''..dataArray['host']..'\'')
  file.writeline('mqtt_port='..dataArray['port'])
  file.writeline('mqtt_topic=\'topic-'..node.chipid()..'\'')
  file.close()
 end

 node.restart()

end

gpio_control.lua:


function gpioControl()
 print('setting GPIO pin: '..dataArray['gpio'])
 
 pin=0
 
 if dataArray['gpio'] == '0' then
  pin=3
 elseif dataArray['gpio'] == '2' then
  pin=4
 end

 if dataArray['state'] == 'on' then
  gpio.write(pin, gpio.HIGH)
 elseif dataArray['state'] == 'off' then
  gpio.mode(pin, gpio.INPUT)
 end

end

init.lua: 
 
require 'config_wifi'
require 'config_mqtt'
require 'bootup'
require 'mqtt_connect'
require 'mqtt_parse'
require 'config_update'
require 'gpio_control'

dataArray={}

bootup()

gpio.mode(3, gpio.INPUT)
gpio.mode(4, gpio.INPUT)

mqttConnect()

 
The WiFi module will try to connect the the WiFi network and from there, to the
MQTT broker. Once connected, it will register to a new topic based on its ID,
which you'll be able to catch in the MQTT broker log file.
 
You'll want to compile all scripts, except for config_wifi.lua, config_mqtt.lua
and init.lua.

I'm still learning Lua, hadn't used it before, which is to say that there's
definitely room for improvement here - however, as they stand, these
scripts do the job in allowing for remote, on demand control!
 
And finally, at this point, you'll be able to publish messages
to your MQTT broker server in order to operate your wifi module:

mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:0;state:on;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:0;state:off;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:2;state:on;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:2;state:off;'

mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:config-update;setting:mqtt;host:192.168.1.3;port:1883;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:config-update;setting:wifi;ssid:wifiHotspot;pass:w1f1P4ss;'

IoT Projects - ESP8266 [programming]

Now all this is left is to program the ESP8266-01 module...!



For this part, I used one of these:

FT232RL 3.3V 5V FTDI USB to TTL Serial Adapter Module for Arduino Mini Port IDXX

Flashing the NodeMCU firmware

1) connect GPIO0 -> GND (Only for when loading the firmware because this pin is what defines the boot mode: firmware load mode or regular mode)

2) git clone https://github.com/themadinventor/esptool.git

3) get your firmware, either from github, the online custom build website from my previous post, etc.

4) sudo python esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash 0x00000 /opt/ESP8266/nodemcu.firmwares/nodemcu_latest.bin

Connecting...
Erasing flash...
Writing at 0x00062000... (100 %)

Leaving...

Writing and Loading Some LUA Scripts

1) remember to disconnect GPIO0 from GND

2) git clone https://github.com/kmpm/nodemcu-uploader.git 

3) sudo ./nodemcu-uploader.py -p /dev/ttyUSB0 -b 115200 upload ../lua.scripts/webserver.lua:webserver.lua [--compile] [--restart]
 
Changing communication to 115200 baud
Preparing esp for transfer.
Transfering webserver.lua as webserver.lua
All done!
 
Using the parameter --compile will trigger lua script compiling at the wifi module:
after uploading the original lua script to the module, the python script will issue
the compile command, generating an lc binary file and deleting the original lua file
on the module.

Using the --restart parameter will issue a module reboot after uploading the script.

IoT Projects - ESP8266 [wiring]

My apologies in advance if the diagram below isn't all that proper, as mentioned before, I'm a bit of a newbie when it comes to wiring circuits...!

Anyways, the important bits to grasp here are:

1) the ESP will handle 3.3v max - going beyond that will most likely damage it
- using a 3.3v voltage regulator to ensure that the voltage being delivered to it is correct

2) the LED strip is represented by the LED on the upper part of the diagram
3) the case fans are represented by the LEDs on the lower part of the diagram
4) a couple of relays are required for wiring the fans and LED strip - and those are 5v relays, so another 5v voltage regulator is required
5) the case fans run at 12v
6) a single power supply is being used: 12v/1A
7) the wifi module's purpose is to control the relays using its two available GPIO pins, by processing MQTT messages from the broker (see previous posts on MQTT and Cubietruck)






Parts list follows:

IKEA Dioder LED strips
5V voltage regulators
3.3V voltage regulators
12V 1.0A Power Supply
5V Relay Module Board Shield
120mm Quiet Case Fans


IoT Projects - ESP8266



The ESP8266-01 Wifi Module is a really cheap component which is programmable and includes two GPIO pins (although it seems you can use the TX/RX pins as GPIO as well and even expand available GPIOs by soldering a few extra pins to the chip legs).

There is quite a community around this module, there are several variants of it - and so far I've been quite impressed by the wifi range on this, as well as the size and features.

First off, there's the official espressif SDK and github resources:

http://espressif.com/new-sdk-release/
https://github.com/esp8266
https://github.com/esp8266/at-command-set


... and then there's NodeMCU:

http://nodemcu.com/index_en.html
https://github.com/nodemcu/nodemcu-firmware
http://www.nodemcu.com/docs/

NodeMCU relies on a Lua interpreter for the applicational part, which makes it easier to code but on the other hand, may require more tweaking with regards to memory usage.

There are however some valuable tricks going with NodeMCU, which was the method I ended up going with on my experiments:

1) lua scripts can be compiled to bytecode files .lua > .lc
2) the NodeMCU firmware build can be customized to include only the modules you'll be needing:

http://frightanic.com/nodemcu-custom-build/

For my solution, I ended up choosing the dev-0.96 github branch and only 7 modules:

node / file / GPIO / WiFi / timer / UART / MQTT

Using the float version rather than integer also seemed to do more for me in terms of system stability.

As for the lua scripts inside the unit, I've managed to secure a couple of interesting features already:

1) MQTT broker reconnect after either wifi or broker outage/recovery
2) ESP settings for wifi and MQTT to be updated over the air (by sending config update messages over MQTT to the wifi module, updating its configuration and rebooting it)

It's equally possible to change timer durations or GPIO operation parameters over MQTT if required - and of course, receive data from the wifi module regarding its status!

IoT Projects - MQTT

As a starting point, I've been looking at a couple of message protocols in order to interact with my embedded devices.

One protocol in particular stood out - MQTT

Basically you'll need to set up a message server (or broker) which will then manage the message queues your clients will be using.

That in turn led me to Mosquitto. Installing Mosquitto on an Ubuntu machine is easy enough, just follow these steps and you're good to go.

Moving on to my embedded boards, my Cubietruck looked like it could handle it - already running Gitlab, Octoprint and a couple more, so why not?

I'm using Cubian, and unfortunately the repo available package for Mosquitto is too old (2012), so I had to install Mosquitto from source:

wget http://mosquitto.org/files/source/mosquitto-1.4.2.tar.gz
sudo apt-get install libc-ares-dev

sudo apt-get install uuid-dev

sudo useradd mosquitto
sudo mkdir /var/log/mosquitto
sudo chown mosquitto:root /var/log/mosquitto/


tar xvzpf mosquitto-1.4.2.tar.gz
cd mosquitto-1.4.2
make clean && make -j3
sudo make install



/etc/mosquitto/mosquitto.conf:


# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

 

/etc/init.d/mosquitto:

#! /bin/sh

### BEGIN INIT INFO
# Provides:        mosquitto
# Required-Start:    $remote_fs $syslog
# Required-Stop:    $remote_fs $syslog
# Default-Start:    2 3 4 5
# Default-Stop:        0 1 6
# Short-Description:    mosquitto MQTT v3.1 message broker
# Description:
#  This is a message broker that supports version 3.1 of the MQ Telemetry
#  Transport (MQTT) protocol.

#  MQTT provides a method of carrying out messaging using a publish/subscribe
#  model. It is lightweight, both in terms of bandwidth usage and ease of
#  implementation. This makes it particularly useful at the edge of the network
#  where a sensor or other simple device may be implemented using an arduino for
#  example.
### END INIT INFO

set -e

PIDFILE=/var/run/mosquitto.pid
DAEMON=/usr/local/sbin/mosquitto

# /etc/init.d/mosquitto: start and stop the mosquitto MQTT message broker

test -x ${DAEMON} || exit 0

umask 022

. /lib/lsb/init-functions

# Are we running from init?
run_by_init() {
    ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
}

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin:/usr/local/sbin"

case "$1" in
  start)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Starting network daemon:" "mosquitto"
    if start-stop-daemon --start --quiet --oknodo --background  --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;
  stop)
    if init_is_upstart; then
        exit 0
    fi
    log_daemon_msg "Stopping network daemon:" "mosquitto"
    if start-stop-daemon --stop --quiet --oknodo --pidfile ${PIDFILE}; then
        log_end_msg 0
        rm -f ${PIDFILE}
    else
        log_end_msg 1
    fi
    ;;


  reload|force-reload)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Reloading network daemon configuration:" "mosquitto"
        if start-stop-daemon --stop --signal HUP --quiet --oknodo --pidfile $PIDFILE; then
            log_end_msg 0
        else
            log_end_msg 1
        fi   
    ;;

  restart)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Restarting network daemon:" "mosquitto"
    if start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile ${PIDFILE}; then
        rm -f ${PIDFILE}
    fi
    if start-stop-daemon --start --quiet --oknodo --background --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;

  try-restart)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Restarting Mosquitto message broker" "mosquitto"
    set +e
    start-stop-daemon --stop --quiet --retry 30 --pidfile ${PIDFILE}
    RET="$?"
    set -e
    case $RET in
        0)
        # old daemon stopped
        rm -f ${PIDFILE}
        if start-stop-daemon --start --quiet --oknodo --background --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
        ;;
        1)
        # daemon not running
        log_progress_msg "(not running)"
        log_end_msg 0
        ;;
        *)
        # failed to stop
        log_progress_msg "(failed to stop)"
        log_end_msg 1
        ;;
    esac
    ;;

  status)
    if init_is_upstart; then
        exit 1
    fi
    status_of_proc -p ${PIDFILE} ${DAEMON} mosquitto && exit 0 || exit $?
    ;;

  *)
    log_action_msg "Usage: /etc/init.d/mosquitto {start|stop|reload|force-reload|restart|try-restart|status}"
    exit 1
esac

exit 0


cubie@Cubian:~$ sudo service mosquitto status
[FAIL] mosquitto is not running ... failed!
cubie@Cubian:~$ sudo service mosquitto start
[ ok ] Starting network daemon:: mosquitto.
cubie@Cubian:~$ sudo service mosquitto status
[ ok ] mosquitto is running.
cubie@Cubian:~$

Cubietruck, ESP8266 and a CTC 3D Printer

Lately I've been messing about with my new CTC 3D printer, but soon figured out that the whole 3D printing set up could do with some upgrades.

Before moving on to what is currently running in the office, I thought I'd break the subjects down a bit:

- Setting up LEDs and case fans on the printer chassis
- Figuring out a way to control them remotely

Since I'm already using a Cubietruck, I thought about using the GPIOs to control these external components. Still, that solution wouldn't be that different from what you can do with a Cubieboard, Cubietruck, Raspberry Pi and similar embedded boards, ranging from 40USD upwards.

So then I found out about the ESP8266-01 wifi module:



This tiny wifi module costs about 2USD (!!!) and while a lot of people consider it mainly as a cheap wifi upgrade for their arduino boards, it can do quite a bit on its own...!

I bought a few of these on Ebay recently, just curious about what I could actually do with them, so in the end, this little guy alone ended up controlling the LEDs and case fans on my 3D printer chassis.

But first off, an important disclaimer:

The electronics on this project was mostly picked up from tips and teachings from a couple of friends, more knowledgeable on the subject.

I'll be posting the several steps in this project on the blog, as soon as I'm able to clean up all the mess around the office...!