Coding Chase - Projects

My 3D Hub

Thank You For Donating:

Ikeeki (linux-sunxi@irc.freenode.net)
Computer Troubleshooters Brasil

Wednesday, 30 December 2015

IoT Projects - ESP8266 [CTC-3D Printer Upgrade] ||

So, after having an ESP8266-01 control my CTC-3D printer LED lighting and Fan cooling, I decided to improve the solution further.

Moving more to ABS than PLA on prints, the fans stopped being that necessary - but what stood out as being a lot more interesting would be to actually turning the printer on or off.

Using the same principles as before, I replaced the Fan part of the solution and added the printer power cord to the available relay.

As an extra, I left cubietruck to manage Octoprint and the remaining solutions in previous posts and set out to build a nice embedded cluster:

  • 4 boards (even cubieboard A10 boards will do for this, but one might consider a cheaper alternative as the Orange Pi perhaps);
  • 4 webcams, in my case, the Hercules Twist HD as it molds perfectly to the case;
  • placing each webcam on the top corners of the printer case, pointing to the heat bed;
  •  Using cubietruck as the manager of the video solution and controlling each of the 4 boards as video feeds;

And since using the command line to issue MQTT commands wasn't actually that practical, I started working on a web platform to manage the solution.

I spent some time working with MEAN.js which I found to be brilliant by the way, integrated MQTT.js and socket.io into the base boilerplate as well as the live video feeds from the webcams.

In the end I got exactly what I wanted:

  • the ESP8266-01 connects to CloudMQTT, a cloud-based, free MQTT broker (moved to it after everything was confirmed to be working properly using mosquitto locally);
  • the MQTT broker is accessible by the MEAN.js web app (which later got published to heroku);
  • the web app uses MQTT.js to subscribe to CloudMQTT and socket.io to perform realtime UI updating if necessary (if I decide to use sensors in the future);
  • the webcams stream the video to the web app.
Using the app, either desktop or tablet or mobile, I'm able to work it:
  1. turn the printer on/off, which saves some power;
  2. turn the lighting on/off, in case there's poor lighting in the room;
  3. use a slicer remotely, like slic3r, generate gcode and upload it to Octoprint;
  4. run the prints remotely and accompany the status using the webcams.
 Some screenshots from the development phase, the web app is very crude, just the buttons and the video streams, just one stream being replicated at the time.

In spite of travelling over the web, the response time with MQTT is lightning fast, basically feels as if a local physical switch is being used...!



 
 As an alternative to MEANjs, there's also meteor.js but I haven't actually looked into that one, apparently it would have saved me some time integrating MQTT an socket.io - perhaps on my next project!
 

CTC-3D and Sitall

I've been rather busy lately but thought I'd document a few relevant pointers regarding 3D printing in general and my CTC-3D printer in particular.

First off, documenting my Octoprint settings for the CTC-3D (at least the relevant bits):










Also, while using the stock firmware + octoprint, I was using ReplicatorG as my slicer software.

After upgrading the firmware, I also moved to Slic3r. ReplicatorG is quite old and not being maintained anymore, apart from the Sailfish variant I guess - whereas Slic3r has many more options and also the possibility of exporting configs, which means I can easily use it on another machine to generate the gcode without having to set it up all over again.

Using slic3r, I just had to make sure to select Makerbot (sailfish) flavor gcode in the settings once I changed firmware.

Second, the fact that after being successful in printing with PLA and ABS using just painter tape, I decided to test printing on glass.

I decided to go with Sitall Glass since the Ebay supplier claimed it can be used without any other adherent substances like ABS goo, glue stick, painter tape, etc. - as a reference, this is seller.

Testing primarily with ABS, I found that the prints would detach from the glass plate mid-print and went on to investigate.

Other than that, the glass seems like it will do the job nicely, it is easily washable and as opposed to the painter tape, the prints won't get line marks on the bottom surface, where the tape strips meet on the plate.

I got myself an infrared thermometer, based on the youtube video below, which was very helpful


What I found out is that, as with in the video, the actual thermistor (thermal sensor) readings on the plate differ significantly from the measurements with the thermometer.

Isolating the printer case side/front panels didn't get it to 110ºC as required by the glass, as I also found out that the stock thermistor underneath the heat bed won't allow such temperatures. Also, the CTC-3D stock firmware is limited to 120ºC max.

After some investigating, two possible solutions were found:

  1. upgrade from stock firmware to sailfish firmware in order to be able to set higher heatbed temperatures, thus increasing the real bed temperature as well, and closer to 110ºC
  2. replace the heatbed thermistor sensor by another one, in order to reduce the temp difference between the one being measured and the actual temp on the bed.

I then set out to replacing the thermistor, based on a 3DHubs user advice as well as upgrade the stock firmware to the Sailfish firmware, in order to be able to set higher heatbed temperatures.

Upgrading the stock firmware wasn't without its hiccups, but no big deal:

[revised from my own post on 3DHubs]

I've recently upgraded my CTC to Sailfish 7.7 (r1432) - I followed the [sailfish firmware install] tutorial, used replicatorg-0040r33-Sailfish-linux.tgz from the thingiverse link as I'm using linux and it worked ok.

A member [on 3DHubs] helped me locate the reset button - it is a hole near the USB port in the back of the printer.

To upgrade the firmware, under linux 64bit, I had to install libusb-0.1-4 : i386 because RepG was reporting errors in the console regarding this lib not being found.

Once RepG was ok, doing the firmware update was as follows in my case:

Used RepG to scan serial devices and got /dev/ttyACM0 detected (the printer)

Machine > Machine Type (Driver) > The Replicator Dual (Sailfish)
Then Upload New Firmware > Makerbot Replicator 1 Single & Dual (v1.5) > Sailfish 7.7 (r1432)


/home/wickwire/Downloads/replicatorg-0040r33-Sailfish/tools/avrdude -C/home/wickwire/Downloads/replicatorg-0040r33-Sailfish/tools/avrdude.conf -cstk500v1 -P/dev/ttyACM0 -b57600 -D -Uflash:w:/home/wickwire/.replicatorg/firmware/mighty_one-Sailfish-v7.7.0-r1432.hex:i -pm1280
avrdude: stk500_recv(): programmer is not responding


As RepG does a few retries before it timed out again in the log, I just pressed the reset button with a pen tip and it started flashing.

I was done in a few seconds (the terminal reported flashing success) - powered down, powered up and the LCD on the printer was working. Going through the options on the LCD to printer info I got the Sailfish info.

Did a factory reset using the printer LCD screen options, rebooted the printer, then checked the printer nozzle offsets and stuff as the guide mentions and all was ok, didn't change anything.

Printed a sample part as follow up using slic3r and being careful to update it to use makerbot (sailfish) instead of makerbot(makerware) and it worked properly.

At this point, the firmware upgrade was OK and printing with ABS on the Sitall glass works, provided I use hairspray for strong adhesion.

The infrared thermometer still registered a 95ºC max temperature on the glass, as I'm still to replace the thermistor.

Removing the heatbed was easy enough and this is what it looks like underneath:



The thermistor is highlighted in green on the photo. To replace it, I was advised to use a 100k ntc, so based on what I was looking at, I went with this model from Ebay (there are several available, mostly for Prusa and RepRap printers, but those are slightly different than what is used on this heatbed).


I'll update the blog with my findings once these parts are delivered.

Sunday, 12 July 2015

IoT Projects - ESP8266 [CTC-3D Printer Upgrade]

Based on the previous blog posts on MQTT, ESP8266-01 and NodeMCU, I decided as mentioned before to upgrade my newly acquired 3D printer, a CTC-3D dual extruder printer.

The printer is awesome and while it already operates remotely via Octoprint on Cubietruck (see previous posts), I did find it useful to add a cooling system to it as well as some lighting inside the chassis, both to be remotely controlled as well.

This post is all about the lua scripts loaded to the ESP8266-01 wifi module, in order to achieve this.

First off, I had several issues with memory usage while using a stock nodeMCU firmware, meaning, a build containing all the available modules.

Trimming the fat from the firmware proved quite valuable in memory terms (check previous posts).

Then, moving on to the lua scripts, I managed to devise a basic management system where I can now update the wifi module configurations for WiFi and MQTT broker over the air, using the MQTT messaging system.

config_wifi.lua:

wifi_ssid='wifiHotspot'
wifi_pass='w1f1P4ss'

config_mqtt.lua:

mqtt_host='192.168.1.2'
mqtt_port=1883
mqtt_topic='topic-'..node.chipid() 
 
bootup.lua:
 
function bootup()
    wifi.setmode(wifi.STATION)
    wifi.sta.config(wifi_ssid,wifi_pass)
end




mqtt_connect.lua:


function mqttActivate()
            m:connect(mqtt_host, mqtt_port, 0, function(conn)
                print("connected to mqtt broker!")
                m:subscribe("/"..mqtt_topic,0, function(conn)
                 tmr.stop(3)
                 tmr.stop(2)
                end)
            end)
end

function mqttConnect()
  m = mqtt.Client(wifi.sta.getmac(), 120, "user", "password")
  m:lwt("/lwt", wifi.sta.getmac(), 0, 0)

  m:on("offline", function(con)
       tmr.alarm(2, 10000, 1, function()
        print ("reconnecting to mqtt broker...")
        mqttActivate()
       end)
  end)

  tmr.alarm(3, 10000, 1, function()
        print("connecting to mqtt broker...")
   mqttActivate()
  end)

  mqttParse()

end

mqtt_parse.lua:


function mqttParse()
  m:on("message", function(conn, topic, data)
    if data ~= nil then
      local i=1
      local offset=0
      while string.find(data,";") do

       field = string.sub(data, offset, string.find(data,":")-1)
       dataArray[field] = string.sub(data, string.find(data,":")+1, string.find(data,";")-1)
       data = string.sub(data, string.find(data, ";") + 1, string.len(data))

       i = i + 1
      tmr.wdclr()
      end

      if dataArray['type'] == 'config-update' then 
       confUpdate()
      elseif dataArray['type'] == 'gpio-control' then
       gpioControl()
      end
    end
  end)
end
 
config_update.lua:

function confUpdate()
 print('changing configuration for module: '..dataArray['setting'])

 if dataArray['setting'] == 'wifi' then
  file.open("config_wifi.lua", "w+")
  file.writeline('wifi_ssid=\''..dataArray['ssid']..'\'')
  file.writeline('wifi_pass=\''..dataArray['pass']..'\'')
  file.close()
 elseif dataArray['setting'] == 'mqtt' then
  file.open("config_mqtt.lua", "w+")
  file.writeline('mqtt_host=\''..dataArray['host']..'\'')
  file.writeline('mqtt_port='..dataArray['port'])
  file.writeline('mqtt_topic=\'topic-'..node.chipid()..'\'')
  file.close()
 end

 node.restart()

end

gpio_control.lua:


function gpioControl()
 print('setting GPIO pin: '..dataArray['gpio'])
 
 pin=0
 
 if dataArray['gpio'] == '0' then
  pin=3
 elseif dataArray['gpio'] == '2' then
  pin=4
 end

 if dataArray['state'] == 'on' then
  gpio.write(pin, gpio.HIGH)
 elseif dataArray['state'] == 'off' then
  gpio.mode(pin, gpio.INPUT)
 end

end

init.lua: 
 
require 'config_wifi'
require 'config_mqtt'
require 'bootup'
require 'mqtt_connect'
require 'mqtt_parse'
require 'config_update'
require 'gpio_control'

dataArray={}

bootup()

gpio.mode(3, gpio.INPUT)
gpio.mode(4, gpio.INPUT)

mqttConnect()

 
The WiFi module will try to connect the the WiFi network and from there, to the
MQTT broker. Once connected, it will register to a new topic based on its ID,
which you'll be able to catch in the MQTT broker log file.
 
You'll want to compile all scripts, except for config_wifi.lua, config_mqtt.lua
and init.lua.

I'm still learning Lua, hadn't used it before, which is to say that there's
definitely room for improvement here - however, as they stand, these
scripts do the job in allowing for remote, on demand control!
 
And finally, at this point, you'll be able to publish messages
to your MQTT broker server in order to operate your wifi module:

mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:0;state:on;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:0;state:off;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:2;state:on;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:gpio-control;gpio:2;state:off;'

mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:config-update;setting:mqtt;host:192.168.1.3;port:1883;'
mosquitto_pub -h 192.168.1.2 -p 1883 -t /topic-12345678 -m 'type:config-update;setting:wifi;ssid:wifiHotspot;pass:w1f1P4ss;'

IoT Projects - ESP8266 [programming]

Now all this is left is to program the ESP8266-01 module...!



For this part, I used one of these:

FT232RL 3.3V 5V FTDI USB to TTL Serial Adapter Module for Arduino Mini Port IDXX

Flashing the NodeMCU firmware

1) connect GPIO0 -> GND (Only for when loading the firmware because this pin is what defines the boot mode: firmware load mode or regular mode)

2) git clone https://github.com/themadinventor/esptool.git

3) get your firmware, either from github, the online custom build website from my previous post, etc.

4) sudo python esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash 0x00000 /opt/ESP8266/nodemcu.firmwares/nodemcu_latest.bin

Connecting...
Erasing flash...
Writing at 0x00062000... (100 %)

Leaving...

Writing and Loading Some LUA Scripts

1) remember to disconnect GPIO0 from GND

2) git clone https://github.com/kmpm/nodemcu-uploader.git 

3) sudo ./nodemcu-uploader.py -p /dev/ttyUSB0 -b 115200 upload ../lua.scripts/webserver.lua:webserver.lua [--compile] [--restart]
 
Changing communication to 115200 baud
Preparing esp for transfer.
Transfering webserver.lua as webserver.lua
All done!
 
Using the parameter --compile will trigger lua script compiling at the wifi module:
after uploading the original lua script to the module, the python script will issue
the compile command, generating an lc binary file and deleting the original lua file
on the module.

Using the --restart parameter will issue a module reboot after uploading the script.

IoT Projects - ESP8266 [wiring]

My apologies in advance if the diagram below isn't all that proper, as mentioned before, I'm a bit of a newbie when it comes to wiring circuits...!

Anyways, the important bits to grasp here are:

1) the ESP will handle 3.3v max - going beyond that will most likely damage it
- using a 3.3v voltage regulator to ensure that the voltage being delivered to it is correct

2) the LED strip is represented by the LED on the upper part of the diagram
3) the case fans are represented by the LEDs on the lower part of the diagram
4) a couple of relays are required for wiring the fans and LED strip - and those are 5v relays, so another 5v voltage regulator is required
5) the case fans run at 12v
6) a single power supply is being used: 12v/1A
7) the wifi module's purpose is to control the relays using its two available GPIO pins, by processing MQTT messages from the broker (see previous posts on MQTT and Cubietruck)






Parts list follows:

IKEA Dioder LED strips
5V voltage regulators
3.3V voltage regulators
12V 1.0A Power Supply
5V Relay Module Board Shield
120mm Quiet Case Fans


IoT Projects - ESP8266



The ESP8266-01 Wifi Module is a really cheap component which is programmable and includes two GPIO pins (although it seems you can use the TX/RX pins as GPIO as well and even expand available GPIOs by soldering a few extra pins to the chip legs).

There is quite a community around this module, there are several variants of it - and so far I've been quite impressed by the wifi range on this, as well as the size and features.

First off, there's the official espressif SDK and github resources:

http://espressif.com/new-sdk-release/
https://github.com/esp8266
https://github.com/esp8266/at-command-set


... and then there's NodeMCU:

http://nodemcu.com/index_en.html
https://github.com/nodemcu/nodemcu-firmware
http://www.nodemcu.com/docs/

NodeMCU relies on a Lua interpreter for the applicational part, which makes it easier to code but on the other hand, may require more tweaking with regards to memory usage.

There are however some valuable tricks going with NodeMCU, which was the method I ended up going with on my experiments:

1) lua scripts can be compiled to bytecode files .lua > .lc
2) the NodeMCU firmware build can be customized to include only the modules you'll be needing:

http://frightanic.com/nodemcu-custom-build/

For my solution, I ended up choosing the dev-0.96 github branch and only 7 modules:

node / file / GPIO / WiFi / timer / UART / MQTT

Using the float version rather than integer also seemed to do more for me in terms of system stability.

As for the lua scripts inside the unit, I've managed to secure a couple of interesting features already:

1) MQTT broker reconnect after either wifi or broker outage/recovery
2) ESP settings for wifi and MQTT to be updated over the air (by sending config update messages over MQTT to the wifi module, updating its configuration and rebooting it)

It's equally possible to change timer durations or GPIO operation parameters over MQTT if required - and of course, receive data from the wifi module regarding its status!

IoT Projects - MQTT

As a starting point, I've been looking at a couple of message protocols in order to interact with my embedded devices.

One protocol in particular stood out - MQTT

Basically you'll need to set up a message server (or broker) which will then manage the message queues your clients will be using.

That in turn led me to Mosquitto. Installing Mosquitto on an Ubuntu machine is easy enough, just follow these steps and you're good to go.

Moving on to my embedded boards, my Cubietruck looked like it could handle it - already running Gitlab, Octoprint and a couple more, so why not?

I'm using Cubian, and unfortunately the repo available package for Mosquitto is too old (2012), so I had to install Mosquitto from source:

wget http://mosquitto.org/files/source/mosquitto-1.4.2.tar.gz
sudo apt-get install libc-ares-dev

sudo apt-get install uuid-dev

sudo useradd mosquitto
sudo mkdir /var/log/mosquitto
sudo chown mosquitto:root /var/log/mosquitto/


tar xvzpf mosquitto-1.4.2.tar.gz
cd mosquitto-1.4.2
make clean && make -j3
sudo make install



/etc/mosquitto/mosquitto.conf:


# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

 

/etc/init.d/mosquitto:

#! /bin/sh

### BEGIN INIT INFO
# Provides:        mosquitto
# Required-Start:    $remote_fs $syslog
# Required-Stop:    $remote_fs $syslog
# Default-Start:    2 3 4 5
# Default-Stop:        0 1 6
# Short-Description:    mosquitto MQTT v3.1 message broker
# Description:
#  This is a message broker that supports version 3.1 of the MQ Telemetry
#  Transport (MQTT) protocol.

#  MQTT provides a method of carrying out messaging using a publish/subscribe
#  model. It is lightweight, both in terms of bandwidth usage and ease of
#  implementation. This makes it particularly useful at the edge of the network
#  where a sensor or other simple device may be implemented using an arduino for
#  example.
### END INIT INFO

set -e

PIDFILE=/var/run/mosquitto.pid
DAEMON=/usr/local/sbin/mosquitto

# /etc/init.d/mosquitto: start and stop the mosquitto MQTT message broker

test -x ${DAEMON} || exit 0

umask 022

. /lib/lsb/init-functions

# Are we running from init?
run_by_init() {
    ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
}

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin:/usr/local/sbin"

case "$1" in
  start)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Starting network daemon:" "mosquitto"
    if start-stop-daemon --start --quiet --oknodo --background  --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;
  stop)
    if init_is_upstart; then
        exit 0
    fi
    log_daemon_msg "Stopping network daemon:" "mosquitto"
    if start-stop-daemon --stop --quiet --oknodo --pidfile ${PIDFILE}; then
        log_end_msg 0
        rm -f ${PIDFILE}
    else
        log_end_msg 1
    fi
    ;;


  reload|force-reload)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Reloading network daemon configuration:" "mosquitto"
        if start-stop-daemon --stop --signal HUP --quiet --oknodo --pidfile $PIDFILE; then
            log_end_msg 0
        else
            log_end_msg 1
        fi   
    ;;

  restart)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Restarting network daemon:" "mosquitto"
    if start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile ${PIDFILE}; then
        rm -f ${PIDFILE}
    fi
    if start-stop-daemon --start --quiet --oknodo --background --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;

  try-restart)
    if init_is_upstart; then
        exit 1
    fi
    log_daemon_msg "Restarting Mosquitto message broker" "mosquitto"
    set +e
    start-stop-daemon --stop --quiet --retry 30 --pidfile ${PIDFILE}
    RET="$?"
    set -e
    case $RET in
        0)
        # old daemon stopped
        rm -f ${PIDFILE}
        if start-stop-daemon --start --quiet --oknodo --background --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON} -- -c /etc/mosquitto/mosquitto.conf ; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
        ;;
        1)
        # daemon not running
        log_progress_msg "(not running)"
        log_end_msg 0
        ;;
        *)
        # failed to stop
        log_progress_msg "(failed to stop)"
        log_end_msg 1
        ;;
    esac
    ;;

  status)
    if init_is_upstart; then
        exit 1
    fi
    status_of_proc -p ${PIDFILE} ${DAEMON} mosquitto && exit 0 || exit $?
    ;;

  *)
    log_action_msg "Usage: /etc/init.d/mosquitto {start|stop|reload|force-reload|restart|try-restart|status}"
    exit 1
esac

exit 0


cubie@Cubian:~$ sudo service mosquitto status
[FAIL] mosquitto is not running ... failed!
cubie@Cubian:~$ sudo service mosquitto start
[ ok ] Starting network daemon:: mosquitto.
cubie@Cubian:~$ sudo service mosquitto status
[ ok ] mosquitto is running.
cubie@Cubian:~$