06 - Android Sensors

Sensors

• Location
  • GPS, Network, Passive
• Sensors
  • Gyroscope, Compass, Orientation, Acceleration
• Camera
  • Intent, Camera API
• Touch
  • Single, Multi
  • Gestures

Location

Most Android devices allow some form of geolocation

• Via Wi-Fi
• Via cell-tower triangulation
• Via GPS

LocationManager class

• Location providers
• Register location update listeners
• Proximity alerts

LocationProvider

• Device might have several
• network - Uses the mobile network or WI-Fi to determine the best location. Might have a higher precision in closed rooms than GPS.
• gps - Use the GPS receiver in the Android device to determine the best location via satellites. Usually better precision than network.
• passive - Allows to participate in location of updates of other components to save energy
• fused - newer google service, combining all providers. Might not work in emulator, force to gps for testing.

• Use Criteria object for flexible selection
• Register LocationListener with LocationManager, to get periodic updates about geoposition

Proximity alert

• Register an Intent
  • Longitude, Latitude and radius
• Alert will be triggered, when device enters the predefined area

GeoCoding

• Geocoder class
  • Get geo-coordinates for given address
  • Get possible address for given geolocation
  • Uses online Google service

Location - Security

• For GPS
  • ACCESS_FINE_LOCATION
• Others
  • ACCESS_COARSE_LOCATION

    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

GPS disabled

• GPS can be disabled by user
• Not possible to activate via software
• Find out via LocationManager – isProviderEnabled() method
• Send the user to the settings via an Intent with the Settings.ACTION_LOCATION_SOURCE_SETTINGS

fun testGps() {
    locationManager = getSystemService(Context.LOCATION_SERVICE) as LocationManager;

    // check if enabled and if not send user to the GSP settings
    // Better solution would be to display a dialog and suggesting to 
    // go to the settings
    if (!locationManager.isProviderEnabled(LocationManager.GPS_PROVIDER)) {
        startActivity(Intent(Settings.ACTION_LOCATION_SOURCE_SETTINGS))
    }
}

GPS

Very battery consuming

Disable GPS updates, when not needed (ie in background)

• onResume()
• onPause()

Last known location

• getLastKnownLocation
• Gives back immediate result
• Can be null

class MapsActivity : AppCompatActivity(), OnMapReadyCallback, LocationListener {

...

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_maps)


    locationManager = getSystemService(Context.LOCATION_SERVICE) as LocationManager
    val criteria = Criteria();
    provider = locationManager.getBestProvider(criteria, false)

    val location = locationManager.getLastKnownLocation(provider)
    if (location != null){
        Log.d("gps", provider)
        onLocationChanged(location)
    } else {
        Log.d("gp","No initial location")
    }

}

Request regular location updates

requestLocationUpdates (String provider, long minTime, float minDistance, LocationListener listener)

• minTime - minimum time interval between location updates, in milliseconds
• minDistance - minimum distance between location updates, in meters
• listener - a LocationListener whose onLocationChanged(Location) method will be called for each location update

override fun onResume() {
    super.onResume()
    if (locationManager != null) {
        locationManager.requestLocationUpdates(
    provider, 400, 1f, this);
    }
}

override fun onPause() {
    super.onPause()
    if (locationManager != null) {
        locationManager.removeUpdates(this);
    }
}

Receive location updates

• Use onLocationChanged to receive position updates
• Mostly requires physical device for any meaningful testing

override fun onLocationChanged(location: Location?) {
    if (mMap != null) {
        mMap.moveCamera(
            CameraUpdateFactory.newLatLng(
                LatLng(
                    location!!.latitude,
                    location!!.longitude
                )
            )
        )
    }
}

Sensors

• Most android systems have sensors for
  • Motion
    • Accelerometer, Gyroscope, Gravity, Rotational vector
  • Position
    • Orientation, Magnetometer
  • Environment
    • Temperature, Pressure, Humidity, Illumination
• Sensor framework
  • Determine availability
  • Sensor capabilities – range, resolution, manufacturer, power usage
  • Acquire raw data, register listeners

Predefined sensor classes

• TYPE_ACCELEROMETER
• TYPE_AMBIENT_TEMPERATURE
• TYPE_GRAVITY
• TYPE_GYROSCOPE
• TYPE_LIGHT
• TYPE_LINEAR_ACCELERATION
• TYPE_MAGNETIC_FIELD
• TYPE_ORIENTATION
• TYPE_PRESSURE
• TYPE_PROXIMITY
• TYPE_RELATIVE_HUMIDITY
• TYPE_ROTATION_VECTOR
• TYPE_TEMPERATURE

Sensor framework

• SensorManager
  • Access and list sensors, register listeners, sensor accuracy, calibration, data rates
• Sensor
  • Instance of specific sensor. Methods for determining sensor capabilities.
• SensorEvent
  • Raw sensor data, type of sensor, accuracy, timestamp
• SensorEventListener
  • Callback interface, receive sensor events

Sensor availability

2 – sensor is deprecated

Sensor manager

• Identifying Sensors
  • Get reference to sensor service

lateinit var sensorManager: SensorManager

// get the sensor manager singleton
sensorManager = getSystemService(Context.SENSOR_SERVICE) as SensorManager

• Get listing of sensors
  • getSensorList(Sensor.TYPE_ALL)
  • TYPE_GYROSCOPE, TYPE_LINEAR_ACCELERATION,TYPE_GRAVITY, ...

var sensorList = sensorManager.getSensorList(Sensor.TYPE_ALL)

• Determine, if there is sensor of certain type
  • getDefaultSensor()
  • There can be several sensors of same type one is designated as default

accSensor = sensorManager.getDefaultSensor(Sensor.TYPE_AMBIENT_TEMPERATURE)
if (accSensor != null){
    ...

Sensor info

There are no requirements from Android, that certain sensors are provided by manufacturers!

Determine the capabilities and attributes of individual sensors

Sensor class

• getResolution()
• getMaximumRange()
• getPower()
• getMinDelay() – microseconds
  • 0 – only events, >0 - streaming
• getVendor()
• getVersion()

fun getSensor():Sensor?{
    var sensor : Sensor? = null;

    if (sensorManager.getDefaultSensor(Sensor.TYPE_GRAVITY) != null){
        val gravSensors = sensorManager.getSensorList(Sensor.TYPE_GRAVITY)

        for (i in 0 until gravSensors.size){
            if (gravSensors[i].vendor.contains("Google") && gravSensors[i].version >= 3){
                sensor = gravSensors.get(i)
            }
        }
        if (sensor == null) {
            sensor = gravSensors[0];
        }
    } else {
        if (sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null) {

            sensor = sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)

        }
    }
    return sensor
}

Sensor Events

Monitoring Sensor Events - SensorEventListener interface

• onAccuracyChanged()
  • SENSOR_STATUS_ACCURACY_LOW, SENSOR_STATUS_ACCURACY_MEDIUM, SENSOR_STATUS_ACCURACY_HIGH, or SENSOR_STATUS_UNRELIABLE.
• onSensorChanged()
  • sensorEvent object contains info about new data
  • Accuracy
  • Sensor
  • Timestap
  • Sensor data reading

Sensor Events, rate

Sensor report rates

• SENSOR_DELAY_NORMAL
  200,000 microseconds
• SENSOR_DELAY_UI
  60,000 microsecond delay
• SENSOR_DELAY_GAME
  20,000 microsecond delay
• SENSOR_DELAY_FASTEST
  0 microsecond delay)
  Or specific delay (api level 11)

class MainActivity : AppCompatActivity(), SensorEventListener {

    lateinit var sensorManager: SensorManager
    lateinit var lightSensor: Sensor

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        // get the sensor manager singleton
        sensorManager = getSystemService(Context.SENSOR_SERVICE) as SensorManager
        lightSensor = sensorManager.getDefaultSensor(Sensor.TYPE_LIGHT)
    }

    override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) {
        // TODO("not implemented") 
    }

    override fun onSensorChanged(event: SensorEvent?) {
        if (event!!.sensor == lightSensor){
            var lux = event!!.values[0]
            textView1.text = "Lux: " + lux.toString()
        }
    }


    override fun onResume() {
        super.onResume()
        sensorManager.registerListener(this, lightSensor, SensorManager.SENSOR_DELAY_NORMAL)
    }

    override fun onPause() {
        super.onPause()
        sensorManager.unregisterListener(this)
    }

}

Sensor - really needed?

• Detecting sensors at runtime
  • Your app might not need all the sensors

lightSensor = sensorManager.getDefaultSensor(Sensor.TYPE_LIGHT)

• Using Google Play filters to target specific sensor configurations
  • When sensor is mandatory

<uses-feature android:name="android.hardware.sensor.accelerometer" android:required="true" /> 

Coordinates

Coordinate system, based on default position

• Tablets are often defaulted to landscape
• Acceleration sensor
• Gravity sensor
• Gyroscope
• Linear acceleration sensor
• Geomagnetic field sensor

Best practices

• You should choose the slowest sampling rate that still meets the needs of your application - System usually provides faster refresh rates
• Use onResume and onPause
• Be aware of power usage
• Sensors are not turned off, when screen turns off
• Detect sensors at runtime and enable or disable application features as appropriate.
• Use Google Play filters to target devices with specific sensor configurations.
• Don't block the onSensorChanged() method
• Verify sensors before you use them (and calibrate)
• DON’T TEST YOUR CODE ONLY ON EMULATOR

Camera

• Most devices have at least one camera
• Most newer devices have 2 – back and front
• Either use existing camera app
  • Intent
• Or use API
  • android.hardware.camera2 API

Camera Intent

private fun dispatchTakePictureIntent() {
    Intent(MediaStore.ACTION_IMAGE_CAPTURE).also { takePictureIntent ->
        takePictureIntent.resolveActivity(packageManager)?.also {
            startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE)
        }
    }
}

Intent based camera usage - MediaStore.ACTION_IMAGE_CAPTURE - MediaStore.EXTRA_OUTPUT with Uri - MediaStore.ACTION_VIDEO_CAPTURE
- startActivityForResult() - onActivityResult()

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

private static final int CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE = 100;
private static final int CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE = 200;

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE) {
        if (resultCode == RESULT_OK) {
            // Image captured and saved to fileUri specified in the Intent
            Toast.makeText(this, "Image saved to:\n" +
                     data.getData(), Toast.LENGTH_LONG).show();
        } else if (resultCode == RESULT_CANCELED) {
            // User cancelled the image capture
        } else {
            // Image capture failed, advise user
        }
    }

    if (requestCode == CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE) {
        if (resultCode == RESULT_OK) {
            // Video captured and saved to fileUri specified in the Intent
            Toast.makeText(this, "Video saved to:\n" +
                     data.getData(), Toast.LENGTH_LONG).show();
        } else if (resultCode == RESULT_CANCELED) {
            // User cancelled the video capture
        } else {
            // Video capture failed, advise user
        }
    }
} 

Toast.makeText(this, "Image saved to:\n" +
        data.getData(), Toast.LENGTH_LONG).show();
// recycle unused bitmaps
if (bitmap != null) {
    bitmap.recycle();
}
stream = getContentResolver().openInputStream(data.getData());
bitmap = BitmapFactory.decodeStream(stream);

imageView.setImageBitmap(bitmap);

Camera API

• Detect and Access Camera
• Create a Preview Class - extend SurfaceView and implement the SurfaceHolder interface
• Build a Preview Layout - create a view layout that incorporates the preview and the user interface controls
• Setup Listeners for Capture - Connect listeners for your interface controls to start image or video capture in response to user actions
• Capture and Save Files
• Release the Camera

Camera manifest

Require camera in manifest

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required=“true" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
~~~~

Or Detect camera

~~~java
/** Check if this device has a camera */
private boolean checkCameraHardware(Context context) {
    if (context.getPackageManager().hasSystemFeature(PackageManager.FEATURE_CAMERA)){
        // this device has a camera
        return true;
    } else {
        // no camera on this device
        return false;
    }
} 

Open camera

• Camera.Open(int)
  • Check exceptions!
• Camera.getParameters()
• Camera.getCameraInfo()

/** A safe way to get an instance of the Camera object. */
public static Camera getCameraInstance(){
    Camera c = null;
    try {
        c = Camera.open(); // attempt to get a Camera instance
    }
    catch (Exception e){
        // Camera is not available (in use or does not exist)
    }
    return c; // returns null if camera is unavailable
} 

Camera surface view

public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback {
    private SurfaceHolder mHolder;
    private Camera mCamera;

    public CameraPreview(Context context, Camera camera) {
        super(context);
        mCamera = camera;
        mHolder = getHolder();
        mHolder.addCallback(this);
        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    }

    public void surfaceCreated(SurfaceHolder holder) {
        try {
            mCamera.setPreviewDisplay(holder);
            mCamera.startPreview();
        } catch (IOException e) {Log.d(TAG, "Error setting camera preview: " + e.getMessage());}
    }

    public void surfaceDestroyed(SurfaceHolder holder) {}

    public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
        if (mHolder.getSurface() == null){return;}
        try {mCamera.stopPreview();} catch (Exception e){}
        try {
            mCamera.setPreviewDisplay(mHolder);
            mCamera.startPreview();

        } catch (Exception e){Log.d(TAG, "Error starting camera preview: " + e.getMessage());}
    }
} 

Preview layout

Place preview into layout

<FrameLayout
    android:id="@+id/camera_preview"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:layout_weight="1"
    /> 

public class CameraActivity extends Activity {

    private Camera mCamera;
    private CameraPreview mPreview;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        // Create an instance of Camera
        mCamera = getCameraInstance();

        // Create our Preview view and set it as the content of our activity.
        mPreview = new CameraPreview(this, mCamera);
        FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview);
        preview.addView(mPreview);
    }
} 

Take picture

• Camera.takePicture()
• Camera.PictureCallback
• Remember to release the camera!

private PictureCallback mPicture = new PictureCallback() {

    @Override
    public void onPictureTaken(byte[] data, Camera camera) {

        File pictureFile = getOutputMediaFile(MEDIA_TYPE_IMAGE);
        if (pictureFile == null){
            Log.d(TAG, "Error creating media file, check storage permissions: " +
                e.getMessage());
            return;
        }

        try {
            FileOutputStream fos = new FileOutputStream(pictureFile);
            fos.write(data);
            fos.close();
        } catch (FileNotFoundException e) {
            Log.d(TAG, "File not found: " + e.getMessage());
        } catch (IOException e) {
            Log.d(TAG, "Error accessing file: " + e.getMessage());
        }
    }
}; 

// Add a listener to the Capture button
Button captureButton = (Button) findViewById(id.button_capture);
captureButton.setOnClickListener(
    new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            // get an image from the camera
            mCamera.takePicture(null, null, mPicture);
        }
    }
); 

Release Camera

    private Camera mCamera;
    private SurfaceView mPreview;
    private MediaRecorder mMediaRecorder;

    ...

    @Override
    protected void onPause() {
        super.onPause();
        releaseMediaRecorder();       // if you are using MediaRecorder, release it first
        releaseCamera();              // release the camera immediately on pause event
    }

    private void releaseMediaRecorder(){
        if (mMediaRecorder != null) {
            mMediaRecorder.reset();   // clear recorder configuration
            mMediaRecorder.release(); // release the recorder object
            mMediaRecorder = null;
            mCamera.lock();           // lock camera for later use
        }
    }

    private void releaseCamera(){
        if (mCamera != null){
            mCamera.release();        // release the camera for other applications
            mCamera = null;
        }
    } 

Camera features

Most can be set using Camera.Parameters

But not all

• Metering and Focus areas
• Face detection
• Time lapse video

Face Detection, Metering Areas, Focus Areas, White Balance Lock, Exposure Lock, Video Snapshot, Time Lapse Video, Multiple Cameras, Focus Distance, Zoom, Exposure Compensation, GPS Data, White Balance, Focus Mode, Scene Mode, JPEG Quality, Flash Mode, Color Effects, Anti-Banding, Picture Format, Picture Size

Camera feature check

Checking feature availability

// get Camera parameters
Camera.Parameters params = mCamera.getParameters();

List<String> focusModes = params.getSupportedFocusModes();
if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) {
  // Autofocus mode is supported
} 

Camera.Parameters object provides a getSupported...(), is...Supported() or getMax...() method to determine if (and to what extent) a feature is supported.

// get Camera parameters
Camera.Parameters params = mCamera.getParameters();
// set the focus mode
params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
// set Camera parameters
mCamera.setParameters(params); 

Touch

• Most mobile devices support touch – single and multi
• View class supports touch events
• The base class for touch support is the MotionEvent class which is passed to Views via the onTouchEvent() method
• To react to touch events you override the onTouchEvent() method
• The MotionEvent class contains the touch related information
  • the number of pointers
  • the X/Y coordinates
  • size and pressure of each pointer
• To react to touch events in an activity, register an OnTouchListener for the relevant Views

Touch events

onTouchEvent

override fun onTouchEvent(event: MotionEvent): Boolean {
    val action: Int = MotionEventCompat.getActionMasked(event)
    return when (action) {
        MotionEvent.ACTION_DOWN -> {
            Log.d("motion", "Action was DOWN")
            true
        }
        MotionEvent.ACTION_MOVE -> {
            Log.d("motion", "Action was MOVE")
            true
        }
        MotionEvent.ACTION_UP -> {
            Log.d("motion", "Action was UP")
            true
        }
        MotionEvent.ACTION_CANCEL -> {
            Log.d("motion", "Action was CANCEL")
            true
        }
        MotionEvent.ACTION_OUTSIDE -> {
            Log.d("motion", "Movement occurred outside bounds of current screen element")
            true
        }
        else -> super.onTouchEvent(event)
    }
}

MultiTouch

• MotionEvent.ACTION_POINTER_DOWN and MotionEvent.ACTION_POINTER_UP are sent starting with the second finger
• For the first finger MotionEvent.ACTION_DOWN and MotionEvent.ACTION_UP are used
• getPointerCount() method on MotionEvent allows you to determine the number of pointers on the device
• To track the touch events from multiple pointers you have to use the MotionEvent.getActionIndex() and the MotionEvent.getActionMasked() methods to identify the index of the pointer and the touch event which happened for this pointer.

Touch - Gestures

• Android provides the GestureDetector class for detecting common gestures.
• Implement GestureDetector.OnGestureListener interface
• Override the View or Activity's onTouchEvent() method, pass along all observed events to the detector instance

private lateinit var mDetector: GestureDetectorCompat

... mDetector = GestureDetectorCompat(this, this)

override fun onTouchEvent(event: MotionEvent): Boolean {
    return if (mDetector.onTouchEvent(event)) {
        true
    } else {
        super.onTouchEvent(event)
    }
}

override fun onShowPress(e: MotionEvent?) {
    TODO("not implemented")
}

override fun onSingleTapUp(e: MotionEvent?): Boolean {
    TODO("not implemented")
}