Introduction

During the summer of 2014, Siddharth Chandra sought out an opportunity to work in the laboratory of Professor Gregory Abowd, a distinguished professor at Georgia Tech’s School of Interactive Computing. Siddharth had read through Professor Abowd’s research and contacted him after he understood Abowd’s research objectives. Professor Abowd allowed Siddharth to work in his lab and collaborate with his PhD and graduate students in order to improve VIBe. Through his experience in the lab, Siddharth was able to create the first wireless VIBe prototype that included an accelerometer.

What Is Virtual Guardian?

The goal of Virtual Guardian software is to use VIBe’s accelerometer to monitor the user in order to determine if the user is experiencing a bout of stimming behavior. Thus, the algorithm needs to be able to use accelerometer data to distinguish between different activities that the user might be involved in. IPAT chose the ADXL345 breakout board from Adafruit and a Arduino UNO as the hardware for this VIBe prototype. Headed by Siddharth Chandra and Russell Pecka(who both have extensive experience in rapid prototyping with Arduinos), the team came up with a flowchart for the first version of the algorithm. This version of the code--a very rudimentary algorithm, included hard coded values buit in and was designed to alert the guardian (in this case by activating an LED) if the accelerometer went over those thresholds. The team realized that such a rigid algorithm would not sufficient because of the individuality of each autistic child

Extension at Georgia Tech

At Georgia Tech, Siddharth set out to make a smarter algorithm, that would be able to adapt to each child, as well as get feedback from the other students in the lab. After his first few days of getting adjusted, Siddharth presented to the lab and was instantly connected with other researchers who were trying to use similar technology to benefit autistic children. Two PhD candidates in Professor Abowd’s laboratory talked to Siddharth and involved him on their projects as well as gave suggestions on his own.

The first, Ivan Riobo was working on a project that was going to utilize Google Glass to benefit caretakers of autistic children. He was using electrodermal activity (EDA), heart rate, and body temperature to extrapolate the emotion of the autistic children--many of whom do not express emotions in the same way that neurotypical children do. By involving Siddharth in his work, Ivan taught Siddharth how to properly design a study that involved autistic children. He made Siddharth understand several issues that he might face as well as solutions to those problems. Siddharth’s work with Ivan were the grounds for the additional research IPAT has completed.

The second researcher, Jonathan Bidwell graciously included Siddharth in his prototyping endeavors as well. Jonathan was attempting to fabricate a bracket to hold several video cameras for capturing night time seizures. This taught Siddharth how to use new equipment as well as gave him access to Georgia Tech’s makerspace.

Building the Prototype and Working With Virtual Guardian

Following this, Siddharth devoted hours of work to building the new VIBe prototype. His first goal was to design a smarter algorithm that would be more proficient in detecting stimming behavior in autistic children. After many revisions, he developed an algorithm that, instead of looking for readings that exceeded a certain threshold, could instantaneously compare readings from the accelerometer. Eventually this algorithm would be able to compare 20 readings from the accelerometer to the previous 20 readings. This allowed Virtual Guardian to detect sudden changes in the behavior for example;. if a child was doing class work and suddenly started rocking back and forth, the sudden change in the movement would be noticed and flagged as an outburst.

Click on any of the Flowcharts below to enlarge them

This diagram illustrates an overview of the logic that the accelerometer follows in Virtual Guardian.

This diagram specifically illustrates the parameters that the algorithm checks and calculates.

The accelerometer will be only one component in the final iteration of Virtual Guardian. We plan to also incorporate a heart rate monitor as well as the function to record the data collected for later analysis.

To effectively test his prototype Siddharth worked on making the accelerometer wireless because it was likely that a hardwired wristband would make autistic children uncomfortable. To do this, Siddharth moved from the Arduino Uno to a much smaller and more compact Teensy board. Utilizing Bluetooth technology, Siddharth was able to create a smaller, more portable device that could be used to gather data and test the new algorithm, shown below.

  /*
This code is the lawful property of International Protective & Assistive Technologies
Copyright © 2014 By Siddharth Chandra All rights reserved.
No part of this publication may be reproduced, 
distributed, or transmitted in any form or by any means, including 
photocopying, recording, or other electronic or mechanical methods, 
without the prior written permission of the publisher.

This code collects and compares the magnitudes of vectors from an accelerometer
and throws flags when the change in magnitude is too great.
*/
#include 
#include 
#include 
#include 

/* Assign a unique ID to this sensor at the same time */
Adafruit_ADXL345 accel = Adafruit_ADXL345(12345);

void setup(void) 
{
  Serial.begin(9600);
  
  // accel.setRange(ADXL345_RANGE_2_G); // sets the range for the accel||movements shouldnt exceed 2G

  /* Initialise the sensor */
  if(!accel.begin())
  {
    /* There was a problem detecting the ADXL345 ... check your connections */
    Serial.println("Ooops, no ADXL345 detected ... Check your wiring!");
    while(1);
  }
}
 
void loop()
{
  XYread();
  mag();
  magDiff();
  //calcTheta();
}

const int arrLeng = 16;

float xArr[arrLeng];
float yArr[arrLeng];

void XYread() //store x values from accel in array
{
  Serial.println("Reading X and Y axis...");
  
  for(int index; index < arrLeng; index++)
  {
   
   sensors_event_t accelEvent;
   accel.getEvent(&accelEvent);
   xArr[index] = accelEvent.acceleration.x; 
   yArr[index] = accelEvent.acceleration.y;
   delay(50);
   
 /* Serial.print("X axis:  ");
   Serial.println(xArr[index]);
   
   Serial.print("Y axis:  ");
   Serial.println(yArr[index]);*/
  }
}

float vecMag[arrLeng];

void mag(){
  Serial.println("MAG");
  
  for(int V; VprevDiff + change || currDiff < prevDiff - change){
    
   prevFlag = currFlag;
   currFlag = millis(); 
    
    if((flag >1 ) && (currFlag - prevFlag) >= interval){
     Serial.println("3 seconds past; flags are reset");
     flag = 0;
   }
   else
   {
     flag++;
    
   Serial.println("   ------   ");
   Serial.print("Flag  #");
   Serial.print(flag);
   Serial.println("   ------   ");
   
   if(flag > alertNum){
     flag=0;
     calcTheta(); 
    }
   }
  }
}





double Tarr[arrLeng];
int STATUS; 

void calcTheta()  //might want to try getting new points before this; data would be atleast a few 100 millis old by now
{

    Serial.println("Starting theta check");
    
    
    while(STATUS<2) //keeps checking theta until 3 seconds have passed with no flag
  {  
    XYread();
    for(int t = 0; t < arrLeng-2; t++)
    {
      Tarr[t] = acos(xArr[t]/vecMag[t]);
      
     /* Serial.print("X component:  ");
      Serial.println(xArr[t]);
      
      Serial.print("Mag:  ");
      Serial.println(vecMag[t]);
      
      Serial.print("Angle:  ");
      Serial.println(Tarr[t]);*/
      TCheck(Tarr[t]);
      
  }
}
}


double TprevRead, TcurrRead;
double Tchange =0.7853;
int Tflag = 0;


long int prevTFlag, currTFlag;


void TCheck(double Read)
{
  
  Serial.println("Checking Theta Readings");
  
  
 
  
  TprevRead = TcurrRead;
 // Serial.println("   ------   ");
  //Serial.print("THETA TprevRead:  ");
  //Serial.println(TprevRead);
  
  TcurrRead = Read;
  //Serial.print("THETA TcurrRead:  ");
  //Serial.println(TcurrRead);
  //Serial.println("   ------   ");
  
  
  //Serial.print("Diff is:  ");
   // Serial.println(TcurrRead - TprevRead);
  
  
  if(TcurrRead> TprevRead + Tchange || TcurrRead < TprevRead - Tchange)
  {
    
    Serial.println("Crossed threshold");
    
    
    prevTFlag = currTFlag;
   currTFlag = millis(); 
    
    if((Tflag >1 ) && (currTFlag - prevTFlag) >= interval)
   {
     STATUS = 2;
     Serial.println("3 seconds past; THETA flags are reset");  // only when time has passed is it time to start over
     Tflag = 0;
     
    // Serial.println(prevTFlag);
     //Serial.println(currTFlag);
     Serial.println("Going back to Mag");
   }
   else if(Tflag > alertNum)
   {
     Serial.println("Moving on to Z axis");
     Tflag = 0;
   }
   else
   {
     Tflag++;
    
   Serial.println("   ------   ");
   Serial.print("THETA Flag  #");
   Serial.print(Tflag);
   Serial.println("   ------   ");
  
   }
  }
}