Android Adventures, part 3: hardware detection in Processing

Go to part 2, part4…

The main reason I got my Android phone was to get Processing running on it, and start grabbing the sensor data.  There were no complete examples  (I could find) on how to actually do this though.  After two days of research I got it working.  Some takeaways:

  • I learned that Processing is ran as an Android “Activity“:  It enters the Activity at the onResume() state, and exits it at the onPause() state.  You need to override these in your sketch to do what you want.   See the above link to a nice image that shows the state tree.
  • All the code you need to author to setup your SensorManagers, SensorEventListeners, and Sensors,  needs to happen in the onResume() function:  Putting this stuff in the sketch’s setup() function won’t work.
  • You can make a single SensorManager, but for each sensor you want to track you need to make a unique SensorEventListener, and Sensor.
  • Once I figured it out I realized how easy it actually is.  Like most things 😉

Here is a list of resources I pulled from, in order of discovery/usefulness:

A (cropped) screenshot off the Phone of the exciting final product!

And the code:

// android_sensorData
// Eric Pavey - 2010-10-10
// http://www.akeric.com
//
// Query the phone's accelerometer and magnetic field data, display on screen.
// Made with Android 2.1, Processing 1.2

//-----------------------------------------------------------------------------------------
// Imports required for sensor usage:
import android.content.Context;
import android.hardware.Sensor;
import android.hardware.SensorEvent;
import android.hardware.SensorManager;
import android.hardware.SensorEventListener;

//-----------------------------------------------------------------------------------------
// Screen Data:
float sw, sh;
// Font Data:
String[] fontList;
PFont androidFont;

// Setup variables for the SensorManager, the SensorEventListeners,
// the Sensors, and the arrays to hold the resultant sensor values:
SensorManager mSensorManager;
MySensorEventListener accSensorEventListener;
MySensorEventListener magSensorEventListener;
Sensor acc_sensor;
float[] acc_values;
Sensor mag_sensor;
float[] mag_values;

//-----------------------------------------------------------------------------------------

void setup() {
  size(screenWidth, screenHeight, A2D);
  sw = screenWidth;
  sh = screenHeight;
  // Set this so the sketch won't reset as the phone is rotated:
  orientation(PORTRAIT);
  // Setup Fonts:
  fontList = PFont.list();
  androidFont = createFont(fontList[0], 16, true);
  textFont(androidFont);
}

//-----------------------------------------------------------------------------------------

void draw() {
  fill(0);
  rect(0,0,sw,sh);
  fill(255);
  if (acc_values != null) {
    text(("Accelerometer: " + acc_values[0] + " " + acc_values[1] + " " + acc_values[2]), 8, 20);
  }
  else {
    text("Accelerometer: null", 8, 20);
  }
  if(mag_values != null) {
    text(("Magnetic Field: " + mag_values[0] + " " + mag_values[1] + " " + mag_values[2]), 8, 40);
  }
  else {
    text("Magnetic Field: null", 8, 40);
  }
}

//-----------------------------------------------------------------------------------------
// Override the parent (super) Activity class:
// States onCreate(), onStart(), and onStop() aren't called by the sketch.  Processing is entered at
// the 'onResume()' state, and exits at the 'onPause()' state, so just override them:

void onResume() {
  super.onResume();
  println("RESUMED! (Sketch Entered...)");
  // Build our SensorManager:
  mSensorManager = (SensorManager)getSystemService(Context.SENSOR_SERVICE);
  // Build a SensorEventListener for each type of sensor:
  magSensorEventListener = new MySensorEventListener();
  accSensorEventListener = new MySensorEventListener();
  // Get each of our Sensors:
  acc_sensor = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);
  mag_sensor = mSensorManager.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD);
  // Register the SensorEventListeners with their Sensor, and their SensorManager:
  mSensorManager.registerListener(accSensorEventListener, acc_sensor, SensorManager.SENSOR_DELAY_GAME);
  mSensorManager.registerListener(magSensorEventListener, mag_sensor, SensorManager.SENSOR_DELAY_GAME);
}

void onPause() {
  // Unregister all of our SensorEventListeners upon exit:
  mSensorManager.unregisterListener(accSensorEventListener);
  mSensorManager.unregisterListener(magSensorEventListener);
  println("PAUSED! (Sketch Exited...)");
  super.onPause();
} 

//-----------------------------------------------------------------------------------------

// Setup our SensorEventListener
class MySensorEventListener implements SensorEventListener {
  void onSensorChanged(SensorEvent event) {
    int eventType = event.sensor.getType();
    if(eventType == Sensor.TYPE_ACCELEROMETER) {
      acc_values = event.values;
    }
    else if(eventType == Sensor.TYPE_MAGNETIC_FIELD) {
      mag_values = event.values;
    }
  }
  void onAccuracyChanged(Sensor sensor, int accuracy) {
    // do nuthin'...
  }
}

Go to part 2, part4…

Android Adventures, part 2
Android Adventures, part 4: vibrate Processing
  • Trackback are closed
  • Comments (24)
  1. Great, referenced, example…..very useful :-)

    • fab
    • January 5th, 2011 7:58am

    thanks Eric, very useful!

    • Lionel
    • February 7th, 2011 4:38am

    THANKS A LOT! I could finally setup processing on my Android up to you.

    just had to add a “redraw();” at the end of the onResume function cause the phone wasn’t showing anything after a pause.

  2. Hey, that redraw() in onResume() fixed a bug I’ve been tracking for a while. Thanks for the tip!

  3. I am sorry Mr. Eric, this is out of topic. But why I cannot find how to send and receive message (SMS) function in Processing for Android? This is very important! I need it. Thanks for your help.

  4. Processing itself will have no libraries for that. You’d need to do it through the Android api itself. As a starting point:
    http://developer.android.com/reference/android/telephony/SmsManager.html

  5. So how to integrate automatic SMS sending with other sensor detection within Processing? Can you give some simple examples? Thank you very much for your help.

  6. I hope that you understand what I mean. With many sensor detection capabilities, Android could become remote sensor gadget and send warning message automatically from remote area. But I don’t understand programming other than Processing itself!

  7. Hi,
    Nice post Eric, but i have a question, how to exploit magnet value, what ‘re the three value ?
    I looked on the doc, on processing forum, but didn’t find explanation…
    Regards Gilles

  8. I was just grabbing them and displaying them as an example. I never put much thought into what they actually mean relative to the phone’s orientation. As far as where the three values come from, I’m not sure what you mean? You can see from the above code sample they are extracted from the SensorEvent that is passed to MySensorEventListener()

    If you want to understand what those values mean relative to the phone, maybe these docs here will help you out:
    http://developer.android.com/reference/android/hardware/SensorEvent.html
    Has a pic showing axis orientation relative to phone, and if you scroll down talks about grabbing data from the different sensor types.

  9. Yeah i need to look your code with more details and your fonction MySensorEventListener()
    Thanks for reply

    • dblack
    • June 5th, 2011 8:34pm

    You are a hero of processing for android. I got some really cool stuff happening on my Galaxy S really quickly by using your example…thanks

  10. Awesome, glad to help :)

    • Rene
    • August 28th, 2011 4:31am

    Great stuff you’ve got here! Keep up the good work :)

    Is it possible to use the magnetic field sensor data with a camera to achieve something like a augmented reality effect? I’ve played around with your code and camera, but can’t seem to figure this thing out. It keeps messing with the camera angels. Any ideas?

    void draw() {
    camera(0, 0, 0, mag_values[0], mag_values[1], 0, 0.0, 1.0, 0.0);

    background(196, 236, 255);
    lights();

    beginShape();
    vertex(-400, 400,-100);
    vertex(-400, -400, -100);
    vertex(400,-400, -100);
    vertex(400,400,-100);
    endShape(CLOSE);
    }

  11. Thanks! Worked straight out of the box so to speak (on Samsung Galaxy S Plus)…now to do something useful with this data 😉 Great work, keep it up!

  12. Thank you very much for posting this good content! I am looking forward to checking out more!

    • André
    • December 6th, 2011 8:20am

    what are the imports for the funccion ellipse and text????

    • Miska Knapek
    • January 20th, 2012 9:23am

    Thanks very much from Helsinki!
    That certainly saved me a lot of headaches :)
    Tried it with an HTC Nexus One, and it ran without issues
    ( well – except the A2D didn’t work, but worked when I wrote “P2D” instead…, for some reason… maybe I set up my environment wrong…)

    /miska

  13. Glad it worked :) I think they may have changed the renderer since I last made these tutorials to make it more consistent when coding.

    • saerdna
    • February 22nd, 2012 5:18am

    for some reason I always get the same error message: (OSX 10.7, processing 2.0a4)

    BUILD FAILED
    /Users/admin/android-sdks/tools/ant/build.xml:602: The following error occurred while executing this line:
    /Users/admin/android-sdks/tools/ant/build.xml:622: Compile failed; see the compiler error output for details.

    Total time: 1 second

  14. When I authored this example, it was on a previous version of Processing, on Windows (not osx). Unfortunately I won’t be much help here with your error, but maybe someone else will have experienced it and provide a solution.

    • tudd
    • April 10th, 2012 8:32am

    This is offtopic, but what about the microphone? I can’t find anything in the developer land for it.

    • Eejin
    • December 5th, 2012 10:50am

    I found out how to run this on processing 2.06b:

    rename
    MySensorEventListener accSensorEventListener;
    MySensorEventListener magSensorEventListener;

    to
    SensorEventListener accSensorEventListener;
    SensorEventListener magSensorEventListener;

    and remove A2D as renderer so something like size(screenWidth, screenHeight);

    • Howie from Shenzhen
    • July 5th, 2013 6:08am

    This is a great intro to AP and sensors. I have a problem importing the Android.hardware.USB package. I get the symbol not found etc error.

    I have the FTDI D311 ev kit which works fine. It acts as a host and exspects the ANDROID to act as an acessory device.. The FTDI examples GPIOdemo…work fine…BUT

    I think it would be great to use processing to develop home controls . How do i get processing to see the Android.hardware.USB package..Android.hardware.Sensor loads OK.

    Thanks

Comment are closed.