06 - Android Sensors

Sensors

Location
- GPS, Network, Passive
Sensors
- Gyroscope, Compass, Orientation, Acceleration
Camera
- Intent, Camera API
Touch
- Single, Multi
- Gestures

Location

Most Android devices allow some form of geolocation

Via Wi-Fi
Via cell-tower triangulation
Via GPS

LocationManager class

Location providers
Register location update listeners
Proximity alerts

LocationProvider

Device might have several
network - Uses the mobile network or WI-Fi to determine the best location. Might have a higher precision in closed rooms than GPS.
gps - Use the GPS receiver in the Android device to determine the best location via satellites. Usually better precision than network.
passive - Allows to participate in location of updates of other components to save energy
fused - newer google service, combining all providers. Might not work in emulator, force to gps for testing.
Use Criteria object for flexible selection
Register LocationListener with LocationManager, to get periodic updates about geoposition

Proximity alert

Register an Intent
- Longitude, Latitude and radius
Alert will be triggered, when device enters the predefined area

GeoCoding

Geocoder class
- Get geo-coordinates for given address
- Get possible address for given geolocation
- Uses online Google service

Location - Security

For GPS
- ACCESS_FINE_LOCATION
Others
- ACCESS_COARSE_LOCATION

    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

GPS disabled

GPS can be disabled by user
Not possible to activate via software
Find out via LocationManager – isProviderEnabled() method
Send the user to the settings via an Intent with the Settings.ACTION_LOCATION_SOURCE_SETTINGS

fun testGps() {
    locationManager = getSystemService(Context.LOCATION_SERVICE) as LocationManager;

    // check if enabled and if not send user to the GSP settings
    // Better solution would be to display a dialog and suggesting to 
    // go to the settings
    if (!locationManager.isProviderEnabled(LocationManager.GPS_PROVIDER)) {
        startActivity(Intent(Settings.ACTION_LOCATION_SOURCE_SETTINGS))
    }
}

GPS

Very battery consuming

Disable GPS updates, when not needed (ie in background)

onResume()
onPause()

Last known location

getLastKnownLocation
Gives back immediate result
Can be null

class MapsActivity : AppCompatActivity(), OnMapReadyCallback, LocationListener {

...

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_maps)

    
    locationManager = getSystemService(Context.LOCATION_SERVICE) as LocationManager
    val criteria = Criteria();
    provider = locationManager.getBestProvider(criteria, false)

    val location = locationManager.getLastKnownLocation(provider)
    if (location != null){
        Log.d("gps", provider)
        onLocationChanged(location)
    } else {
        Log.d("gp","No initial location")
    }

}

Request regular location updates

requestLocationUpdates (String provider, long minTime, float minDistance, LocationListener listener)

minTime - minimum time interval between location updates, in milliseconds
minDistance - minimum distance between location updates, in meters
listener - a LocationListener whose onLocationChanged(Location) method will be called for each location update

override fun onResume() {
    super.onResume()
    if (locationManager != null) {
        locationManager.requestLocationUpdates(
	provider, 400, 1f, this);
    }
}

override fun onPause() {
    super.onPause()
    if (locationManager != null) {
        locationManager.removeUpdates(this);
    }
}

Receive location updates

Use onLocationChanged to receive position updates
Mostly requires physical device for any meaningful testing

override fun onLocationChanged(location: Location?) {
    if (mMap != null) {
        mMap.moveCamera(
            CameraUpdateFactory.newLatLng(
                LatLng(
                    location!!.latitude,
                    location!!.longitude
                )
            )
        )
    }
}

Sensors

Most android systems have sensors for
- Motion
  - Accelerometer, Gyroscope, Gravity, Rotational vector
- Position
  - Orientation, Magnetometer
- Environment
  - Temperature, Pressure, Humidity, Illumination
Sensor framework
- Determine availability
- Sensor capabilities – range, resolution, manufacturer, power usage
- Acquire raw data, register listeners

Predefined sensor classes

TYPE_ACCELEROMETER
TYPE_AMBIENT_TEMPERATURE
TYPE_GRAVITY
TYPE_GYROSCOPE
TYPE_LIGHT
TYPE_LINEAR_ACCELERATION
TYPE_MAGNETIC_FIELD
TYPE_ORIENTATION
TYPE_PRESSURE
TYPE_PROXIMITY
TYPE_RELATIVE_HUMIDITY
TYPE_ROTATION_VECTOR
TYPE_TEMPERATURE

Sensor framework

SensorManager
- Access and list sensors, register listeners, sensor accuracy, calibration, data rates
Sensor
- Instance of specific sensor. Methods for determining sensor capabilities.
SensorEvent
- Raw sensor data, type of sensor, accuracy, timestamp
SensorEventListener
- Callback interface, receive sensor events

Sensor availability

Sensors

2 – sensor is deprecated

Sensor manager

Identifying Sensors
- Get reference to sensor service

lateinit var sensorManager: SensorManager

// get the sensor manager singleton
sensorManager = getSystemService(Context.SENSOR_SERVICE) as SensorManager

Get listing of sensors
- getSensorList(Sensor.TYPE_ALL)
- TYPE_GYROSCOPE, TYPE_LINEAR_ACCELERATION,TYPE_GRAVITY, ...

var sensorList = sensorManager.getSensorList(Sensor.TYPE_ALL)

Determine, if there is sensor of certain type
- getDefaultSensor()
- There can be several sensors of same type one is designated as default

accSensor = sensorManager.getDefaultSensor(Sensor.TYPE_AMBIENT_TEMPERATURE)
if (accSensor != null){
    ...

Sensor info

There are no requirements from Android, that certain sensors are provided by manufacturers!

Determine the capabilities and attributes of individual sensors

Sensor class

getResolution()
getMaximumRange()
getPower()
getMinDelay() – microseconds
- 0 – only events, >0 - streaming
getVendor()
getVersion()

fun getSensor():Sensor?{
    var sensor : Sensor? = null;

    if (sensorManager.getDefaultSensor(Sensor.TYPE_GRAVITY) != null){
        val gravSensors = sensorManager.getSensorList(Sensor.TYPE_GRAVITY)

        for (i in 0 until gravSensors.size){
            if (gravSensors[i].vendor.contains("Google") && gravSensors[i].version >= 3){
                sensor = gravSensors.get(i)
            }
        }
        if (sensor == null) {
            sensor = gravSensors[0];
        }
    } else {
        if (sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null) {
            
            sensor = sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)

        }
    }
    return sensor
}

Sensor Events

Monitoring Sensor Events - SensorEventListener interface

onAccuracyChanged()
- SENSOR_STATUS_ACCURACY_LOW, SENSOR_STATUS_ACCURACY_MEDIUM, SENSOR_STATUS_ACCURACY_HIGH, or SENSOR_STATUS_UNRELIABLE.
onSensorChanged()
- sensorEvent object contains info about new data
- Accuracy
- Sensor
- Timestap
- Sensor data reading

Sensor Events, rate

Sensor report rates

SENSOR_DELAY_NORMAL
200,000 microseconds
SENSOR_DELAY_UI
60,000 microsecond delay
SENSOR_DELAY_GAME
20,000 microsecond delay
SENSOR_DELAY_FASTEST
0 microsecond delay)
Or specific delay (api level 11)

class MainActivity : AppCompatActivity(), SensorEventListener {

    lateinit var sensorManager: SensorManager
    lateinit var lightSensor: Sensor

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        // get the sensor manager singleton
        sensorManager = getSystemService(Context.SENSOR_SERVICE) as SensorManager
        lightSensor = sensorManager.getDefaultSensor(Sensor.TYPE_LIGHT)
    }

    override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) {
        // TODO("not implemented") 
    }

    override fun onSensorChanged(event: SensorEvent?) {
        if (event!!.sensor == lightSensor){
            var lux = event!!.values[0]
            textView1.text = "Lux: " + lux.toString()
        }
    }


    override fun onResume() {
        super.onResume()
        sensorManager.registerListener(this, lightSensor, SensorManager.SENSOR_DELAY_NORMAL)
    }

    override fun onPause() {
        super.onPause()
        sensorManager.unregisterListener(this)
    }

}

Sensor - really needed?

Detecting sensors at runtime
- Your app might not need all the sensors

lightSensor = sensorManager.getDefaultSensor(Sensor.TYPE_LIGHT)

Using Google Play filters to target specific sensor configurations
- When sensor is mandatory

<uses-feature android:name="android.hardware.sensor.accelerometer" android:required="true" /> 

Coordinates

Coordinate system, based on default position

Tablets are often defaulted to landscape
Acceleration sensor
Gravity sensor
Gyroscope
Linear acceleration sensor
Geomagnetic field sensor

Best practices

You should choose the slowest sampling rate that still meets the needs of your application - System usually provides faster refresh rates
Use onResume and onPause
Be aware of power usage
Sensors are not turned off, when screen turns off
Detect sensors at runtime and enable or disable application features as appropriate.
Use Google Play filters to target devices with specific sensor configurations.
Don't block the onSensorChanged() method
Verify sensors before you use them (and calibrate)
DON’T TEST YOUR CODE ONLY ON EMULATOR

Camera

Most devices have at least one camera
Most newer devices have 2 – back and front
Either use existing camera app
- Intent
Or use API
- android.hardware.camera2 API

Camera Intent

private fun dispatchTakePictureIntent() {
    Intent(MediaStore.ACTION_IMAGE_CAPTURE).also { takePictureIntent ->
        takePictureIntent.resolveActivity(packageManager)?.also {
            startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE)
        }
    }
}

Intent based camera usage

MediaStore.ACTION_IMAGE_CAPTURE
- MediaStore.EXTRA_OUTPUT with Uri
MediaStore.ACTION_VIDEO_CAPTURE
startActivityForResult()
onActivityResult()

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

private static final int CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE = 100;
private static final int CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE = 200;

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE) {
        if (resultCode == RESULT_OK) {
            // Image captured and saved to fileUri specified in the Intent
            Toast.makeText(this, "Image saved to:\n" +
                     data.getData(), Toast.LENGTH_LONG).show();
        } else if (resultCode == RESULT_CANCELED) {
            // User cancelled the image capture
        } else {
            // Image capture failed, advise user
        }
    }

    if (requestCode == CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE) {
        if (resultCode == RESULT_OK) {
            // Video captured and saved to fileUri specified in the Intent
            Toast.makeText(this, "Video saved to:\n" +
                     data.getData(), Toast.LENGTH_LONG).show();
        } else if (resultCode == RESULT_CANCELED) {
            // User cancelled the video capture
        } else {
            // Video capture failed, advise user
        }
    }
} 

Toast.makeText(this, "Image saved to:\n" +
        data.getData(), Toast.LENGTH_LONG).show();
// recycle unused bitmaps
if (bitmap != null) {
    bitmap.recycle();
}
stream = getContentResolver().openInputStream(data.getData());
bitmap = BitmapFactory.decodeStream(stream);

imageView.setImageBitmap(bitmap);

Camera API

Detect and Access Camera
Create a Preview Class - extend SurfaceView and implement the SurfaceHolder interface
Build a Preview Layout - create a view layout that incorporates the preview and the user interface controls
Setup Listeners for Capture - Connect listeners for your interface controls to start image or video capture in response to user actions
Capture and Save Files
Release the Camera

Camera manifest

Require camera in manifest

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required=“true" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />

Or Detect camera

/** Check if this device has a camera */
private boolean checkCameraHardware(Context context) {
    if (context.getPackageManager().hasSystemFeature(PackageManager.FEATURE_CAMERA)){
        // this device has a camera
        return true;
    } else {
        // no camera on this device
        return false;
    }
} 

Open camera

Camera.Open(int)
- Check exceptions!
Camera.getParameters()
Camera.getCameraInfo()

/** A safe way to get an instance of the Camera object. */
public static Camera getCameraInstance(){
    Camera c = null;
    try {
        c = Camera.open(); // attempt to get a Camera instance
    }
    catch (Exception e){
        // Camera is not available (in use or does not exist)
    }
    return c; // returns null if camera is unavailable
} 

Camera surface view

public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback {
    private SurfaceHolder mHolder;
    private Camera mCamera;

    public CameraPreview(Context context, Camera camera) {
        super(context);
        mCamera = camera;
        mHolder = getHolder();
        mHolder.addCallback(this);
        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    }

    public void surfaceCreated(SurfaceHolder holder) {
        try {
            mCamera.setPreviewDisplay(holder);
            mCamera.startPreview();
        } catch (IOException e) {Log.d(TAG, "Error setting camera preview: " + e.getMessage());}
    }

    public void surfaceDestroyed(SurfaceHolder holder) {}

    public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
        if (mHolder.getSurface() == null){return;}
        try {mCamera.stopPreview();} catch (Exception e){}
        try {
            mCamera.setPreviewDisplay(mHolder);
            mCamera.startPreview();

        } catch (Exception e){Log.d(TAG, "Error starting camera preview: " + e.getMessage());}
    }
} 

Preview layout

Place preview into layout

<FrameLayout
    android:id="@+id/camera_preview"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:layout_weight="1"
    /> 

public class CameraActivity extends Activity {

    private Camera mCamera;
    private CameraPreview mPreview;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        // Create an instance of Camera
        mCamera = getCameraInstance();

        // Create our Preview view and set it as the content of our activity.
        mPreview = new CameraPreview(this, mCamera);
        FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview);
        preview.addView(mPreview);
    }
} 

Take picture

Camera.takePicture()
Camera.PictureCallback
Remember to release the camera!

private PictureCallback mPicture = new PictureCallback() {

    @Override
    public void onPictureTaken(byte[] data, Camera camera) {

        File pictureFile = getOutputMediaFile(MEDIA_TYPE_IMAGE);
        if (pictureFile == null){
            Log.d(TAG, "Error creating media file, check storage permissions: " +
                e.getMessage());
            return;
        }

        try {
            FileOutputStream fos = new FileOutputStream(pictureFile);
            fos.write(data);
            fos.close();
        } catch (FileNotFoundException e) {
            Log.d(TAG, "File not found: " + e.getMessage());
        } catch (IOException e) {
            Log.d(TAG, "Error accessing file: " + e.getMessage());
        }
    }
}; 

// Add a listener to the Capture button
Button captureButton = (Button) findViewById(id.button_capture);
captureButton.setOnClickListener(
    new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            // get an image from the camera
            mCamera.takePicture(null, null, mPicture);
        }
    }
); 

Release Camera

    private Camera mCamera;
    private SurfaceView mPreview;
    private MediaRecorder mMediaRecorder;

    ...

    @Override
    protected void onPause() {
        super.onPause();
        releaseMediaRecorder();       // if you are using MediaRecorder, release it first
        releaseCamera();              // release the camera immediately on pause event
    }

    private void releaseMediaRecorder(){
        if (mMediaRecorder != null) {
            mMediaRecorder.reset();   // clear recorder configuration
            mMediaRecorder.release(); // release the recorder object
            mMediaRecorder = null;
            mCamera.lock();           // lock camera for later use
        }
    }

    private void releaseCamera(){
        if (mCamera != null){
            mCamera.release();        // release the camera for other applications
            mCamera = null;
        }
    } 

Camera features

Most can be set using Camera.Parameters

But not all

Metering and Focus areas
Face detection
Time lapse video

Face Detection, Metering Areas, Focus Areas, White Balance Lock, Exposure Lock, Video Snapshot, Time Lapse Video, Multiple Cameras, Focus Distance, Zoom, Exposure Compensation, GPS Data, White Balance, Focus Mode, Scene Mode, JPEG Quality, Flash Mode, Color Effects, Anti-Banding, Picture Format, Picture Size

Camera feature check

Checking feature availability

// get Camera parameters
Camera.Parameters params = mCamera.getParameters();

List<String> focusModes = params.getSupportedFocusModes();
if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) {
  // Autofocus mode is supported
} 

Camera.Parameters object provides a getSupported...(), is...Supported() or getMax...() method to determine if (and to what extent) a feature is supported.

// get Camera parameters
Camera.Parameters params = mCamera.getParameters();
// set the focus mode
params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
// set Camera parameters
mCamera.setParameters(params); 

Touch

Most mobile devices support touch – single and multi
View class supports touch events
The base class for touch support is the MotionEvent class which is passed to Views via the onTouchEvent() method
To react to touch events you override the onTouchEvent() method
The MotionEvent class contains the touch related information
- the number of pointers
- the X/Y coordinates
- size and pressure of each pointer
To react to touch events in an activity, register an OnTouchListener for the relevant Views

Touch events

Touch

onTouchEvent

override fun onTouchEvent(event: MotionEvent): Boolean {
    val action: Int = MotionEventCompat.getActionMasked(event)
    return when (action) {
        MotionEvent.ACTION_DOWN -> {
            Log.d("motion", "Action was DOWN")
            true
        }
        MotionEvent.ACTION_MOVE -> {
            Log.d("motion", "Action was MOVE")
            true
        }
        MotionEvent.ACTION_UP -> {
            Log.d("motion", "Action was UP")
            true
        }
        MotionEvent.ACTION_CANCEL -> {
            Log.d("motion", "Action was CANCEL")
            true
        }
        MotionEvent.ACTION_OUTSIDE -> {
            Log.d("motion", "Movement occurred outside bounds of current screen element")
            true
        }
        else -> super.onTouchEvent(event)
    }
}

MultiTouch

MotionEvent.ACTION_POINTER_DOWN and MotionEvent.ACTION_POINTER_UP are sent starting with the second finger
For the first finger MotionEvent.ACTION_DOWN and MotionEvent.ACTION_UP are used
getPointerCount() method on MotionEvent allows you to determine the number of pointers on the device
To track the touch events from multiple pointers you have to use the MotionEvent.getActionIndex() and the MotionEvent.getActionMasked() methods to identify the index of the pointer and the touch event which happened for this pointer.

Touch - Gestures

Android provides the GestureDetector class for detecting common gestures.
Implement GestureDetector.OnGestureListener interface
Override the View or Activity's onTouchEvent() method, pass along all observed events to the detector instance

private lateinit var mDetector: GestureDetectorCompat

... mDetector = GestureDetectorCompat(this, this)

override fun onTouchEvent(event: MotionEvent): Boolean {
    return if (mDetector.onTouchEvent(event)) {
        true
    } else {
        super.onTouchEvent(event)
    }
}

override fun onShowPress(e: MotionEvent?) {
    TODO("not implemented")
}

override fun onSingleTapUp(e: MotionEvent?): Boolean {
    TODO("not implemented")
}

Sensors​

Location​

LocationProvider​

Proximity alert​

GeoCoding​

Location - Security​

GPS disabled​

GPS​

Last known location​

Request regular location updates​

Receive location updates​

Sensors​

Predefined sensor classes​

Sensor framework​

Sensor availability​

Sensor manager​

Sensor info​

Sensor Events​

Sensor Events, rate​

Sensor - really needed?​

Coordinates​

Best practices​

Camera​

Camera Intent​

Camera API​

Camera manifest​

Open camera​

Camera surface view​

Preview layout​

Take picture​

Release Camera​

Camera features​

Camera feature check​

Touch​

Touch events​

onTouchEvent​

MultiTouch​

Touch - Gestures​

Sensors

Location

LocationProvider

Proximity alert

GeoCoding

Location - Security

GPS disabled

GPS

Last known location

Request regular location updates

Receive location updates

Sensors

Predefined sensor classes

Sensor framework

Sensor availability

Sensor manager

Sensor info

Sensor Events

Sensor Events, rate

Sensor - really needed?

Coordinates

Best practices

Camera

Camera Intent

Camera API

Camera manifest

Open camera

Camera surface view

Preview layout

Take picture

Release Camera

Camera features

Camera feature check

Touch

Touch events

onTouchEvent

MultiTouch

Touch - Gestures