Tagged: tech notes Toggle Comment Threads | Keyboard Shortcuts

  • julie 4:00 pm on March 17, 2016 Permalink
    Tags: , , tech notes   

    Raspberry Pi 3B setup 

    install NOOBS from https://www.raspberrypi.org/downloads/noobs/ v 1.8.0 released 2016-02-29

    Get online using wifi dongle and QMUL Guest login (fuck eduroam for now)

    >sudo apt-get update

    >sudo apt-get install mysql-server

    >sudo apt-get upgrade

    >sudo pip install PyMySQL

    Changed mind and went for MariaDB as that is what I use on PEW
    >sudo apt-get install mariadb-server
    As MySQL was already installed this process migrates MySQL dbs etc to MariaDB and removes old MySQL stuff.

    Set-up remote access:
    VNC
    >sudo apt-get install tightvncserver
    >tightvncserver
    — Start a VNC server from the terminal: This example starts a session on VNC display one (:1) with full HD resolution:

    >vncserver :1 -geometry 1920×1080 -depth 24

    NOTE: With RPi3 and NOOBS 29-2-2016 DO NOT RUN the stop service as described on the RPi website as you’ll lose the GUI desktop you are working on… >service lightdm stop

    After this I couldn’t access the Pi from my mac like this – cmd-k from Finder and try to login vnc://PI-IPADDRESS :1
    Probably something to do with the university network bullshit.
    (automation set-up/start on boot instructs are here: https://www.raspberrypi.org/documentation/remote-access/vnc/README.md)

    Decided to try weaved:

    Weaved Connectd https://www.weaved.com
    >create account on https://www.weaved.com/
    >sudo apt-get install weavedconnectd

    To set-up SSH access:
    >sudo weavedinstaller
    — enter account details (email and passw)
    Then Option 1
    Option 1 again
    Option 1 again (SSH)
    yes to default port num
    Enter name
    Exit

    The above ref’d from: https://www.weaved.com/installing-weaved-raspberry-pi-raspbian-os/

    For VNC access via Safari on MacOSX:
    Check tightvnc is running (if not start a server as above)
    >ps ax | grep tightvnc
    Display what port(s) tightvnc is listening to
    >`sudo netstat -apn | grep tcp | grep LISTEN
    >
    Pick lowest port number and install a new Weaved Service Attachment for VNC using the default port of PORTNUMBER.
    >sudo weavedinstaller
    — enter account details (email and passw)
    Then Option 1
    Option 3 (you can change Port number if default isn’t what you want)
    Enter name
    Exit

    Sign in to Weaved account in Safari at weaved.com
    Your connections will be listed.
    Click on the one you want and follow instructs. You’ll need your TightVNC password.
    et voila.

    Generate key to SSH into pew:
    > ssh-keygen -t rsa

    Copy public key to clipboard
    > pbcopy < ~/.ssh/id_rsa.pub Add public key to PEW in the authorized_keys file in .ssh ----- Change hostname, first replace the default name in here: >sudo nano /etc/hostname

    ++ new name nmrpi ++

    Then find the line starting with 127.0.0.1, and change the name following it to your new hostname:
    > sudo nano /etc/hosts
    —–
    Find MAC address:
    > ifconfig

    The HWaddr is the MAC address
    —-

     
  • julie 2:40 pm on January 7, 2016 Permalink
    Tags: app development, tech notes   

    LiveCode 

    Source: Products | LiveCode

    App development resource based in Edinburgh.

     
  • julie 1:19 pm on November 25, 2015 Permalink
    Tags: tech notes   

    Resilience & Longevity of NMR system 

    When I leave..?

    Data Storage
    Access to Data
    Updating the records

     
  • julie 12:24 pm on November 25, 2015 Permalink
    Tags: HPC, tech notes   

    Intro to High Performance Computing (HPC) course 

    Taught by Lukas in eecs IT (25 nov 2015)

    http://www.eecs.qmul.ac.uk/~lukas/downloads/IntroductionToHPC.pdf
    Apocrita

    MPI – message passing interface

    Use Screen for qlogin sessions, incase your SSH connection goes, then your process will continue.

    • Resource Allocation slide

    Core = slot in SGE
    smp (this gives you slots on the same machine will not spread over nodes)
    if you specify 2 slots and 2 GB, you’ll end up being allocated 4GB – 2GB per slot.

    • Other HPC facilities

    You can use Warwick (they have a few nodes with GPUs) – request an account from QMUL
    archer.ac.uk UK super computer facility
    prace-ri is an EU cluster, need to justify the use of the facility

    • NMR NOTES

    NMR data – in which ways can it be sub-divided for parallelised? Where are the dependencies?
    Python – sort out GitHub committing with SourceTree (first remove the sensitive details into an external file)

     
  • julie 12:44 pm on November 23, 2015 Permalink
    Tags: , , omega, tech notes   

    NMR tracking – Omega 

    started: 2015-11-20 11:18:00.854
    data dump: 2015-11-23 10:49:05.320

    44015 rows

     
  • julie 4:19 pm on November 20, 2015 Permalink
    Tags: , , , , tech notes   

    1st test – NMR tracking in lab (local version) 

    Today the iMac was deployed to track colony Omega (20 NMRs) using 7 sensors. The data is being stored on a local MySQL db.

    deployed code on github (misslake)

    estimated data logging (as determined with lights on):
    24 rows of non-blank data per 30 seconds
    scales up to around 70,000 per day if NMR movement frequency maintains this pace
    files size: MySQL dump of 41,000 rows = 1.6MB. scales up to around 40MB per million rows
    14.28 days per 40MB/million rows

    System left on, with display sleep (NOT system sleep).

    NMR-colony-omega-map

     
  • julie 9:07 pm on November 17, 2015 Permalink
    Tags: , , , tech notes   

    Local iMac db setup 

    status at 20151117:
    Python script ok. Failing at the MySQL commit as the primary key stuff is not letting the data into the table.
    To do: add data to the sensor table. check PKs and FKs. check them again. Look at using python to check for duplicates instead of doing it this way.

     
  • julie 3:23 pm on October 1, 2015 Permalink
    Tags: mariaDB, , , tech notes   

    Setting up iMac (OSXYosemite 10.10.5) 

    $ sudo easy_install pip
    $ sudo pip install -U setuptools
    $ sudo pip install pyserial

    install MariaDB as that is what is installed on pew
    ==

    • make sure Xcode is uptodate or install it

    $ xcode-select –install

    • install homebrew

    $ ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”

    • Check Homebrew

    $ brew doctor

    • Update Homebrew

    $ brew update

    • Check MariaDB version in brew repo

    $ brew info mariadb

    • Install MariaDB

    $ brew install mariadb

    • Run the Database Installer (change the version number in the path to match where MariaDB was installed in previous step)

    $ unset TMPDIR
    $ cd /usr/local/Cellar/mariadb/10.0.21/
    $ mysql_install_db

    • Start MariaDB

    $ mysql.server start

    • Secure the Installation

    $ mysql_secure_installation

    • Connect to MariaDB

    $ mysql -u root -p

    • Verify MariaDB Version

    $ select @@version;

    Fabulous source of the above instructions – https://mariadb.com/blog/installing-mariadb-10010-mac-os-x-homebrew

    ===

    • $ sudo pip install MySQL-python

    $ sudo pip install PyMySQL

    ===

    • plug in USB to serial convertor and a sensor. You may have FTDI drivers installed – check using from root dir:

    $ ls /dev/cu.*

    You are looking for /dev/cu.usbserial or /dev/tty.usbserial (see http://pbxbook.com/other/mac-tty.html )

    If not showing up install driver at http://www.prolific.com.tw/US/ShowProduct.aspx?p_id=229&pcid=41 and restart.

    ===

    • Change path in conn.py to be correct serial port path:

    ser0 = serial.Serial(‘/dev/tty.usbserial’, 9600, timeout=5)

    • and fuck me this all worked straight away. I am learning after all.

    Update 13th Nov 2015
    To get up and running on the command line:
    $ mysql.server start
    then
    $ mysql -u root -p
    created nmr db on iMac and created tables using latest schema hosted on github.

    TODO – export database from pew and install on imac so that nmr_index contains a tag ID that it can match.

     
  • julie 9:56 am on September 18, 2015 Permalink
    Tags: RHEL, tech notes   

    Server Maintenance (RHEL, pew) 

    Update and Upgrade software:

    $ sudo yum update

    $ sudo yum upgrade

    OR, use the below is you can’t be arsed to review and accept changes (not recommended):

    $ sudo yum -y update && yum -y upgrade

    Check services are running – all should be active (running) and enabled to start on boot:

    $ service httpd status
    $ service mariadb status
    $ service firewalld status

    List all zones:

    $ sudo firewall-cmd –permanent –zone=public –list-all

    RHEL7: How to get started with Firewalld.

     
  • julie 5:23 pm on September 15, 2015 Permalink
    Tags: tech notes   

    Rubular: regular expression editor and tester 

    Rubular is a Ruby-based regular expression editor and tester. It’s a handy way to test regular expressions as you write them. Rubular is an especially good fit for Ruby and Rails developers, since it uses Ruby on the server to evaluate regexes, but should also be useful for those working in other programming languages and frameworks (Java, PHP, Python, Javascript, etc.)

    Source: Rubular: a Ruby regular expression editor and tester

     
  • julie 9:42 am on September 9, 2015 Permalink
    Tags: , , tech notes   

    Soft robots 

    • Hardware:

    Calibration button for hardware – bumper switch

    • Software:

    Arduino
    Speed – can change the freq, by not using PWM. accelstepper – library to control acceleration.
    3 motors ok with arduino – more motors might be tricky if added due to the switching speed. Might be best to have an arduino per motor, or per pair.

    Interupts so that if the motor hits the switch the programme will shut down. This is initiated at setup and sits in the background.

    Moulds / 3D printing:

    Silicone objects:

     
  • julie 11:59 am on September 3, 2015 Permalink
    Tags: tech notes   

    sepans Cheats – sepans 

    A cheat sheet containing scripts that I normally use (shell, sql, python .etc)

    Source: sepans Cheats – sepans

    contains some nice text sorting reg ex bits

     
  • julie 11:23 am on August 12, 2015 Permalink
    Tags: longevity, tech notes   

    Making the work survive the long-term / error handling 

    Notes:

    Long form work needs to have plans to keep it running (ref Ken Goldberg’s MementoMori, and LongPlayer and Walter de Maria https://en.wikipedia.org/wiki/New_York_Earth_Room)

    New NMRs and new tags – add system to pick up new tag and log it [detect if tag is in tag db, of not add it. check Primary Key status as the nmr_data will not accept a tag ID is it isn’t in nmr_tag_id table]

     
  • julie 11:26 am on July 29, 2015 Permalink
    Tags: , tech notes   

    Python tips 

    Creating the code on Mac OS, to switch to Pi later

    ensure pip install and upgraded
    $ sudo pip install -U setuptools
    $ sudo pip install pyserial
    $ brew install mysql-connector-c ( on Mac OSX you need the mysql connection lib, will be installed if XCode installed apparently, see http://stackoverflow.com/questions/16182294/installing-mysql-python-on-mac-os-x)
    $ sudo pip install MySQL-python

    $ sudo pip install PyMySQL [alternative called pymysql which is a Python client connection to MySQL , follows same API as MySQL-python and works with Python 3 – this works so far…]

    Can’t TCP into the server db so use SSH
    sudo pip install paramiko [which also installs pycrypto and ecdsa]

    >> import serial
    >> import MySQLdb

    For Pi:

    1. apt-get install python-mysqldb
     
  • julie 10:25 am on July 24, 2015 Permalink
    Tags: , tech notes   

    Raspberry Pi set-up (old) 

    On mac, Download NOOBS from https://www.raspberrypi.org/downloads/noobs/
    Unzip

    Format the microSD using SDFormatter software (I’ve used a 32GB card as I want to be able to backup the NMR data locally, just-in-case. This could be done to a USB stick too I guess).

    Copy files from root of NOOBS to freshly formatted SD card.

    Put card in Raspi and power up. Follow install instructs.

    To enter networking details use Run: wpa_gui BUT I have always had issues when doing this – unless your credentials are perfect the network is likely to mess up, and you’ll need to reset the wpa_supplicant.conf file. ALWAYS make a copy of a working version before editing it, manually or via the GUI.

    Making a copy of a file:

    Always – Update and Upgrade

    • sudo apt-get update
    • sudo apt-get upgrade

    Which version of Debian running?
    $ lsb_release -a

    • Generate SSH key on Pi and add public key to pew.
    • Add laptop public key to pi for ssh without passwrd
    • Install tightvncserver in case I need to access the GUI on the pi (i.e. for node-red)

    Setting up a light sensor on GPIO pins HEREHERE

    This is for PEW I think:

    To ensure start-up of these services on reboot (ensure you use the .service):

    $ sudo systemctl enable firewalld.service
    $ sudo systemctl enable mariadb.service
    $ sudo systemctl enable httpd.service

    Other commands:

    $ sudo systemctl status firewalld
    $ sudo systemctl start firewalld
    $ sudo systemctl stop firewalld

    • set-up static IP

    http://thepihut.com/blogs/raspberry-pi-tutorials/16683276-how-to-setup-a-static-ip-address-on-your-raspberry-pi

    access port 3306 on pew through frank
    $ ssh -NfL 9000:pew.eecs.qmul.ac.uk:3306 USERNAMEHERE@frank.eecs.qmul.ac.uk

    Set-up Python:
    $ sudo apt-get install python-setuptools

     
  • julie 2:16 pm on July 10, 2015 Permalink
    Tags: tech notes   

    AVID minitrackers commands 

    C [67] = continuous read mode ON
    c [99] = continuous read mode OFF
    B [66] = beep ON
    b [98] = beep OFF
    [32] = toggle read/stop [will need to send this periodically (every 30 secs?) to stop reader getting ‘stuck’
    $ [36] = 3 min warning beep OFF

    1. [35] = 3 min warning beep ON

    NOTE: numbers in [ ] are the ascii codes for the characters.

     
  • julie 11:34 am on July 10, 2015 Permalink
    Tags: tech notes   

    Firewalld / iptables 

    on pew, i removed the iptables and engaged firewalld. These are firewalls that protect the server from unwanted incoming connections.

    Check status (you cannot run iptables AND firewalld, but firewalld is recommended now to usurp iptables)
    $ systemctl status iptables

    Disable and mask iptables
    $ sudo systemctl stop iptables
    $ sudo systemctl mask iptables

    Enable firewalld

    $ systemctl enable firewalld

    Start firewalld

    $ systemctl start firewalld

    open port for db

    http://www.tecmint.com/configure-firewalld-in-centos-7/2/

     
  • julie 12:50 pm on July 8, 2015 Permalink
    Tags: , tech notes   

    MySQL TIPS 

    for pew:

    • sudo mysql (to give permissions)
    • create database ; (only needs setting up once)
    • use ; (before you start working on a db, make sure you are using the right one)
    • create table
      …etc see below.
    • show tables;
    • describe
      ;

    DB: nmr
    TABLE: datatest

    • CREATE TABLE datatest (time TIMESTAMP, recID TINYINT(10), avidID VARCHAR(16));

    Check status (on pew, RHEL)
    > mysqladmin -u root -p status

    Start Mariadb (on pew, RHEL)
    > sudo systemctl start mariadb

    Deleting rows
    > delete from nmr_records where id = >”799″;

    > delete from nmr_index where colony = “800”;

    Add columns:
    > alter table nmr_records add column worker_pc smallint(4);

     
  • julie 4:52 pm on July 7, 2015 Permalink
    Tags: command line, tech notes   

    Terminal TIPS (linux) 

    This is a mix of Mac command line, RHEL, and Raspian commands — not all work on all platforms.

    $ sudo find / -name XXX [Find things]

    $ sudo -i [elevate your self to super user]

    $ cd / [change to root directory]

    $ cd ~ [change to home directory]

    $ ls -a [list everything folder AND files]

    $ cat > filename.txt [use this to make a text, or any, file, Ctrl D to save and exit text edit]

    $ cat filename.txt [use this to read the file]

    $ cat /etc/redhat-release [to get OS version of RedHat]

    $ hostnamectl [will return hostname, machine ID, kernel and OS info]

    $ ip addr show | grep inet [will show ip details, not sure why one below is diff]

    $ curl -s checkip.dyndns.org | sed -e ‘s/.*

    Current IP Address:

    $ df –show you used and available space on the harddisk

    $ shutdown -h now =shutdown now
    $ shutdown -h +10 =shutdown in 10 mins

    $ rm -i filename.txt = remove file but get confirmation request before it goes

    $ rm -r example = removes directory (name: example) and all files and subdirs in it

    $ cp -p file1 file2 =copy file keeping mode, ownership and timestamp

    $ cp -i file1 file2 =copy file1 to file2 i If file2 exists get confirmation before overwriting

    $ mv -i file1 file2 =rename file1 to file2 (+ get confirmation if file2 exists)

    1. mkdir /u01 = mount a file system, by first creating a directory and mount it as below.
    1. mount /dev/sdb1 /u01

    $ locate filename =locate a file

    $ tail -f /var/log/syslog [will show you live log activity] on Pi anyway *******THIS

    More useful tips: http://www.thegeekstuff.com/2010/11/50-linux-commands/
    Also: http://www.tecmint.com/51-useful-lesser-known-commands-for-linux-users/

    Generate key:
    $ ssh-keygen -t rsa

    Copy public key to clipboard:

    $ pbcopy < ~/.ssh/id_rsa.pub

    help brings up the OS version:

    GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)

    File Permissions:

    • 777 (rwxrwxrwx) No restrictions on permissions. Anybody may do anything. Generally not a desirable setting.
    • 755 (rwxr-xr-x) The file’s owner may read, write, and execute the file. All others may read and execute the file. This setting is common for programs that are used by all users.
    • 700 (rwx——) The file’s owner may read, write, and execute the file. Nobody else has any rights. This setting is useful for programs that only the owner may use and must be kept private from others.
    • 666 (rw-rw-rw-) All users may read and write the file.
    • 644 (rw-r–r–) The owner may read and write a file, while all others may only read the file. A common setting for data files that everybody may read, but only the owner may change.
    • 600 (rw——-) The owner may read and write a file. All others have no rights. A common setting for data files that the owner wants to keep private.

    Directory Permissions:

    • 777 (rwxrwxrwx) No restrictions on permissions. Anybody may list files, create new files in the directory and delete files in the directory. Generally not a good setting.
    • 755 (rwxr-xr-x) The directory owner has full access. All others may list the directory, but cannot create files nor delete them. This setting is common for directories that you wish to share with other users.
    • 700 (rwx——) The directory owner has full access. Nobody else has any rights. This setting is useful for directories that only the owner may use and must be kept private from others.

    install phpmyadmin

    sudo yum install http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm
    sudo yum install phpmyadmin
    edit conf files to enable remote access (might have to fiddle with path name):
    sudo nano /etc/httpd/conf.d/phpMyAdmin.conf

    src: http://www.tecmint.com/install-lamp-in-centos-7/

     
  • julie 3:46 pm on July 6, 2015 Permalink
    Tags: nemacode, tech notes   

    Naming:
    Nemacode – serverside
    Critter – data vis / public facing

    Today:
    upgrade node-red to 0.10.10
    start Python build, set up GIT on Pi

     
  • julie 3:57 pm on June 9, 2015 Permalink
    Tags: tech notes   

    Build your own Data Cloud – QMUL

    UPLOAD SLIDES

    MapReduce / Hadoop on RaspberryPi

    If your data is processed once, maybe not keep it in the HFDS as it is slow(?) check. HDFS (Hadoop Distributed File SYstem).

    Name Nodes store the metadata
    Data Nodes store data (nodes are replicated a number of times (3 usually)

    In nano hadoop-2.6.0/etc/hadoop/hadoop-env.sh (configuration file)
    Edit the java implementation to use.
    export JAVA_HOME=${JAVA_HOME}
    to
    export JAVA_HOME=”/home/pi/ejdk1.8.0_33/linux_armv6_vfp_hflt/jre/” [or equiv]

    edit core-site.xml

    fs.defaultFS
    hdfs://pi-0:9000 <--- this is the master node. you have to place the tag inside the

    edit hdfs-site.xml
    dfs.replication
    1
    dfs.namenode.name.dir
    /home/pi/had-hdfs/
    dfs.datanode.data.dir
    /home/pi/had-hdfs/

    edit mapred-site.xml <--- may already be correctly edited mapreduce.framework.name
    yarn

    Yarn
    – hadoop-2.6.0/etc/hadoop/yarn-site.xml

    yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.hostname
    localhost
    <---- localhost uses localhost as master. for NME ask ITS for the master IP on their Hadoop set-up  You need to start Hadoop process form the Master node... You need to start Yarn process form the Master node... [so both of these will be running at QMUL already] This was a SHIT workshop. to unzip tars in command line: tar xzvf [filename]

     
  • julie 3:53 pm on June 2, 2015 Permalink
    Tags: tech notes   

    Installing software on the RPi 2 for testing the receivers:

    Install node and node-red
    $ curl -sL https://deb.nodesource.com/setup | sudo bash –

    $ sudo apt-get install -y build-essential python-dev python-rpi.gpio nodejs

    $ sudo npm install -g –unsafe-perm node-red

    src [http://nodered.org/docs/hardware/raspberrypi.html]

    Mainly using node-red in the first instance to test the serial port stuff as it’s so quick and easy. Will end up using python for the final version.

    Working out power for all 7 readers:
    Tested in the lab with a variable power supply.
    each reader uses about 300mAh at 9v (they are usually powered by a 9v alkaline battery)
    For 7 I need a 3A 9V plug-in power supply, and can then wire they all together into one source.
    Ordered one plus a load of speaker cable to extend the power of each reader.

     
  • julie 5:40 pm on May 27, 2015 Permalink
    Tags: , tech notes   

    Hardware set-up today

    Download Raspian
    Install raspian on the RPi
    Install wifi dongle
    Problems getting onto Eduroam.

    Will need to get the Pi on the network with a fixed IP

     
  • julie 8:57 am on May 27, 2015 Permalink
    Tags: , tech notes   

    Tutorial – Learn Python in 10 minutes – Stavros’ Stuff.

    PySerial

    python tutorials and resources

     
  • julie 6:51 pm on May 19, 2015 Permalink
    Tags: tech notes   

    Data Storage at QMUL – meeting with Tim Kay.
    Agreed that MySQL will probably be adequate for my needs. We calculated that the basic data (ID, scannerID, timestamp) would total about 10 Gb over 10 years. The data can be hosted on EECS as the whole lot will eventually be centralised and stored at IBM, so SBCS will have access in the future.
    I can serve webpages for the front end, and will be able to use QMUL authentication for a web form that can be used to store the NMR measurements and so on, also accessible by Chris and his team.
    I can have a separate domain name, but this may expire – would be better to have something that qmul will keep going … I like The Naked Mole Rat Diaries… and could buy a ten year domain, but need to think about this further.
    Also should be fine to run a live stream of the data through an API (I will need this for my work anyway).
    Regarding data privacy we think there is no issue as no personal (human) details stored.
    Consider having a back-up / mirror of the data too (i.e. Amazon Cloud if they can give a free academic license? and can they provide a real-time feed?)

    Powering the receivers – Ho suggests we wire them together and use a single power block that gives us 9v. We’re going to rig a few up and see what power they require on continuous read in the lab. Need to check that the current will be enough for all of them.
    Ho also has an old UPS that I can have for the project – it needs new batteries, but that should be a cheaper option than getting a new one. Need to check if it is noisy for the NMRs.

    Ordered the serial hub, power surge extension with USB and other bits. The hub will power the RPi, or it can be plugged directly into the power extension. This might be best. Also gives me some 5v ports to play with for any additional sensors.

    Software will be written in Python, need to look into the local DB i.e.: https://oss.oetiker.ch/rrdtool/

    NMR colony 800 map 20150519v1

     
  • julie 12:56 pm on May 13, 2015 Permalink
    Tags: tech notes   

    OpenSensors.io training 

    IoT with OpenSensors – 28 April 2015

    IoT deep dive with OpenSensors https://opensensors.io/

    Node-RED – plumbing system for iot. nodered.org (opensource but developed at IBM Emerging Technology)

    @BorisAdryan

    node.js based webservice on various platfroms – Rpi, mac, cloud…

    • website with GUI
    • large library
    • flows shared as JSON strings

    http://flows.nodered.org

    Node.js is a prerequirement, then run

    > sudo npm install -g node-red

    > node-red

     

    Access the interface at http://localhost:1880/

    About 100 extra nodes on npmjs

    IBM have a cloud version on Bluemix https://console.ng.bluemix.net

    • If the serial port is picking up extra broadcasts from other Pi’s:

    sudo nano /etc/inittab

    comment out the line (using a hash) with ttyAMA0

    sudo nano /boot/cmdline

    delete the reference to ttyAMA0

    ====

    REFS & TERMS:

    Data communication protocol – MQTT.org

    MQTT is designed for M2M, connects via TCP, very lightweight for sending small packets of data. HTTP has big headers.

    OpenSensors is a MQTT broker. Each part of the system is autonomous – the sensor sending doesn’t care where the data is going, the broker doesn’t care where data is from, the user … etc.

    Notes to create basic node-red io’s: http://www.slideshare.net/BorisAdryan/node-red-coursecamjamjuly2014

    Payload – is the thing/data you are sending (can be JSON, PDF, whatever)

    COAP – HTTP lite, avoid headers but use GET and POST  HTTP-like requests. http://coap.technology/

    opensensors.io

    user: misslake pp: usual33 e: misslake@lubric.com

    • client ID: 1321. pp: 8SbXhPdT

    SSE’s  (which OpenSensors uses) get through firewalls more easily than websockets. RESEARCH this.

    ===

    Machine Learning

    supervised learning – classification and regression

    dimensionality reduction

    deep learning

    unsupervised learning – xxx and xxx

    hierarchical clustering (split into pairs and see what we end up with)

    • build distance matrix on basis of correlation coefficient (cc)
    • build tree

    k-means clustering

    • finds the optimum clustering for you, starting from a set of n starting conditions.
    • K is optimum number of clusters in the set

    Support vector machine – train the data, get suggestion for grouping, present new data

    Random forests – find the shortest tree that gets me to my data (classifications)

    Outlier prediction through regression

    cassandra – open sensors stores data.

    For longevity, in 10 years say…

    • will the data from inferior hardware still be comparable to the latest precision hardware?
    • give all your data as much metadata as possible for longevity (hardware versions, softwares, etc)

    The concrete embedded sound sculpture… layers under layers under layers. Bio, concrete, plastic, PLA, metal, alloys, … The time is now. write up with Gavin. Climate change. Disposible culture. Noise pollution. Wifi. Electronic material world. Nature as technology. (+ collaborator from materials lab? Zoe XX?)

    Yodit is sending notes on Python data analysis (uses iPython)

    $ killall -# screen   (where # is the number of screens open)

    NMRTracker Flow

    [{“id”:”2b3c023e.d4c3fe”,”type”:”serial-port”,”serialport”:”/dev/cu.usbserial”,”serialbaud”:”9600″,”databits”:”8″,”parity”:”none”,”stopbits”:”1″,”newline”:”A”,”bin”:”false”,”out”:”char”,”addchar”:true},{“id”:”be7e25f7.4181d8″,”type”:”debug”,”name”:””,”active”:true,”console”:”false”,”complete”:”payload”,”x”:624,”y”:142,”z”:”afb085ff.504f78″,”wires”:[]},{“id”:”68e6ebb4.971914″,”type”:”serial in”,”name”:”minitracker”,”serial”:”2b3c023e.d4c3fe”,”x”:127,”y”:165,”z”:”afb085ff.504f78″,”wires”:[[“6c493c2d.93b6c4”]]},{“id”:”6c493c2d.93b6c4″,”type”:”function”,”name”:”str.search”,”func”:”var str = msg.payload;\nvar n = str.search(\”806\”);\nif (n == -1){\n    msg.payload = \”nmr01\”;\n} else {\n    msg.payload = \”nmr02\”;    \n}\nreturn msg;”,”outputs”:1,”valid”:true,”x”:320,”y”:188,”z”:”afb085ff.504f78″,”wires”:[[“d960f09e.269f1”]]},{“id”:”d960f09e.269f1″,”type”:”delay”,”name”:””,”pauseType”:”delay”,”timeout”:”1″,”timeoutUnits”:”seconds”,”rate”:”1″,”rateUnits”:”second”,”randomFirst”:”1″,”randomLast”:”5″,”randomUnits”:”seconds”,”drop”:false,”x”:477,”y”:234,”z”:”afb085ff.504f78″,”wires”:[[“be7e25f7.4181d8”]]}]

    NMRTracker Flow 20150508

     
  • julie 9:41 am on April 28, 2015 Permalink
    Tags: tech notes   

    IoT deep dive with OpenSensors https://opensensors.io/

    Node-RED – plumbing system for iot. nodered.org
    @BorisAdryan

    node.js based webservice on various platfroms – Rpi, mac, cloud…

    • website with GUI
    • large library
    • flows shared as JSON strings
     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel