SM App structure

๐Ÿ—‚ Folder structure

๐Ÿš› Activities



  • Display camera preview
  • Take photos
  • Discover/connect to servers
  • Manage settings

Noteworthy code:

  1. hideStatusAndActionBars() โ†’ hide status/action bars
  2. setContentView() โ†’ load layout and views
  3. verifyPermissions() โ†’ verifies all permissions
    • read/write external storage
    • access wifi state
    • internet
    • camera
  4. setupSharedPref() โ†’ initializes the PreferenceManager
  5. createDeviceID() โ†’ creates unique ID for the device when using the app for the first time
  6. initViews() โ†’ assigns views to variables
  7. setViewListeners() โ†’ sets listeners
  8. start FragmentHandler
  9. start ServerConnection
  10. start CameraInstance
  1. sets isCapturing = true
  2. extracts bitmap from camera preview (CameraInstance)
  3. converts image to greyscale (PhotoConverter)
  4. sends image to server (Http)
  5. listens to server response (โ†’ sendBitmap())

Loads all images from the app directory into memory so that GalleryFragment doesnโ€™t lag when opened up the first time

  • changes colour and text of the mSelectDeviceButton based on the connection status (ServerConnection.isConnectedToServer):
    • true: green, โ€œHOSTNAMEโ€
    • false: red, โ€œnot connectedโ€
  • sets isCapturing = false
  • shows toast to the user, informing them that the matching permission was denied
onMatchResult(matchID, img)



  • Display the cropped screenshot that was returned from the server
  • optional: download and display full screenshot when the user requests it
  • share/save screenshots
  • return to CameraActivity

Noteworthy code:

  1. get serverURL/matchID/images from Intent
  2. check whether it received a cropped image
    • if not: enter fullScreenshotOnlyMode
      • no switching between screenshots
      • saving both images not allowed
  3. downloadFullScreenshotInThread()
  1. starts download of full screenshot in a new thread
  2. โ†’ onScreenshotDownloaded()
  3. saves screenshot to app directory, displays it on ImageView
  1. saves the currently displayed image to the phone gallery
  2. hasSharedImage = true โ†’ when closing the activity, the image(s) will also be saved to the app storage โ†’ accessed by GalleryFragment
  1. only executed when the cropped and full screenshot are available
  2. saves both images to phone gallery
  3. hasSharedImage = true โ†’ when closing the activity, both images will also be saved to the app storage โ†’ accessed by GalleryFragment
  1. opens android share context menu, allowing users to share the currently displayed screenshot
  2. hasSharedImage = true โ†’ when closing the activity, both images will also be saved to the app storage โ†’ accessed by GalleryFragment

๐Ÿš— Fragments

๐Ÿ”„ Rotation Fragments



= abstract class that extends Fragment and adds extra functionality allowing the fragment to change orientation without defining an extra layout

  • rotation works by:
    1. removing the fragment (โ†’ removeThisFragmentForRotation())
    2. getting a rotated view of the fragment (โ†’ rotateView())
    3. attaching the rotated fragment
Noteworthy code:
rotateView(rotationDeg: Int, v: View)

called when the view gets created (or reloaded), returns view thats rotated by rotationDeg degrees


called when the view should be rotated (detach, rotate, attach), removes the view without an animation

removeThisFragment(removeBackground: Boolean)

called when the fragment should actually be closed (not rotated)

  • removes the dark background of the fragment (if removeBackground = true)
  • plays animation


  • display all available servers
  • allow users to connect to servers on the list by clicking on them
Noteworthy code:
  1. gets server list from ServerConnection object in the CameraActivity
  2. (if not null): adds all servers from the list to the mServerList variable
  3. (else): retry after 100ms
  1. initializes all views
  2. attaches ArrayAdapter that fills up the ListView with the elements from mServerList
  3. sets onClickListener that listens to clicks on items of the ListView


  • show a list of all cropped (and full) screenshots the user has taken and then either saved/shared in the ResultsActivity
  • allow users to click on screenshots โ†’ GalleryPreviewFragment
Noteworthy code:
  1. initializes all view variables
  2. attaches GridBaseAdapter to the GridView


  • allow users to save/share previously matched screenshots like in ResultsActivity
Noteworthy code:

similar to ResultsActivity

  • retrieves the provided image files from the bundle
  • increments numberOfAvailableImages for each image retrieved

โธ Fixed Fragments


  • notify the user when there is not match result
  • allow user to open the feedback fragment
  • allow user to request and show the full screenshot


  • allow user to send feedback to a server


  • allow user to modify app preferences (currently only matching mode)
  • the preference layout (and the keys to access the values) are stored in โ€œ/res/xml/rood_preferences.xmlโ€
List of preferences keys:
  • settings_algorithm_key: โ€œALG_MODE_KEYโ€ (stored in โ€œ/values/strings.xmlโ€, access via getString(@StringRes int resId))
  • settings_logging_key: โ€œLOG_MODE_KEYโ€ (stored in โ€œ/values/strings.xmlโ€, accessed via getString(@StringRes int resId))
  • get algorithm mode key: MATCHING_MODE_PREF_KEY = getString(R.string.settings_algorithm_key)
Accessing preferences:
  1. get default shared preferences via: val sp = PreferenceManager.getDefaultSharedPreferences(context: Context)
  2. access the preferences via key value principle e.g.: sp.getBoolean(PREFERENCE_KEY_HERE: String, defaultValue: Boolean)
Noteworthy code:
  • loads root_preferences layout which lets users change predefined preferences

๐Ÿ‘‹ Helpers


Noteworthy code:
openFragment(fragment: Fragment, containerID: Int, transition: Int? = null)
  • attaches the provided fragment in the given container (with an animation)
  1. removes the GalleryFragment if it is open (removeThisFragmentForRotation())
  2. re-attaches (openFragment())
  3. re-attaches the GalleryPreviewFragment if it was open


  • rescale a photo and convert it to grayscale


  • provide various utility functions
Noteworthy code:
CompareSizesByArea: Comparator
  • comparator class used by CameraInstance to compare camera preview sizes

๐Ÿ”— Network



  • start network discovery if not connected to a server, update mServerUrlList
  • send heartbeats to a server if connected to one
  • update UI in CameraActivity accordingly
Noteworthy code:
mHandler: Handler
discoverRunnable: Runnable
onServerURLsGet(servers: List<Pair<String, String>>)
  • if the returned list is not empty:
    • updates serverUrlList and checks if mServerURL is in the list
    • if it is: change connection to connected (โ†’ onConnectionChanged(true))
onConnectionChanged(isConnected: Boolean)
heartbeatRunnable: Runnable
setServerUrl(hostname: String)
  • sets mServerURL to an url that belongs to the hostname provided in the parameters via hostname: String



  • provide functions for discovering servers in the local network

Noteworthy code:

  • todo: @TF



Noteworthy code:

sendBitmap(bitmap, serverURL, activity, matchingOptions, permissionToken)
  1. encodes bitmap (= camera image) to a base64 string and appends it to JSON
  2. appends matchingOptions, deviceName, deviceID to JSON
  3. adds permissionToken, if not empty, to JSON
  4. sends request, waits for these possible responses:
    • error โ†’ call onPermissionDenied
    • response with url param โ€œerrorโ€ = โ€œpermissionRequiredโ€ โ†’ requestPermission()
    • response with url param โ€œhasResultโ€ = true โ†’ decode base64 to bitmap call onMatchResult()
    • result with url param โ€œhasResultโ€ = false โ†’ start ErrorFragment
sendHeartbeatRequest(serverURL, activity)
  1. makes GET request to the servers heartbeat route
  2. if it returns a response, do nothing
  3. if it returns an error, call onHeartbeatFail
requestPermission(bitmap, serverURL, activity, matchingOptions)
  1. @TF

๐Ÿ‘ Views



  • provide a helper class for the CameraActivity that works abstracts the Camera 2 API:
    • initializing the TextureView
    • choosing a fitting preview size
    • setting up camera outputs
    • listen to camera state callbacks

Noteworthy code:

  • initializes the TextureView found in the CameraActivity layout
  • sets callback object that listens to changes of the TextureView
    • onSurfaceTextureAvailable = the starting point of the application code
openCamera(width: Int, height: Int)
  1. setupCameraOutputs() โ†’ obtained mPreviewSize
  2. checks for CAMERA permission
  3. opens camera using CameraManager, mCameraDeviceStateCallback as callback
setupCameraOutputs (width: Int, height: Int)
  • helper function for determining the optimal camera output size (= preview size)
  • checks each output resolution for each camera of the device and picks the biggest one with a 4:3 ratio โ†’ mPreviewSize

= a callback object for listening to camera device state changes: - onOpened() โ†’ createCameraCaptureSession - onDisconnected() โ†’ closes cameraDevice - onError() โ†’ finishes CameraActivity

  1. creates a Surface from mTextureView
  2. chooses this Surface as target for displaying the camera preview
  3. creates a CaptureRequest, mCameraDeviceStateCallback as callback object

= a callback object for listening to CaptureSession callbacks - onConfigured() - โ†’ sets a repeating request on the CameraSession - โ†’ preview gets displayed on the Surface of mTextureView - onConfigureFailed() โ†’ throws RuntimeException



Noteworthy code:

getView(position: Int, convertView: View, parent: ViewGroup)
  • returns a View based on the provided position
  • sets an onClickListener on each returned View โ†’ onClick: openGalleryPreviewFragment(firstImageFile, secondImageFile) of CameraActivityFragmentHandler
  • private helper class for storing the information about which images are stored in a returned View

๐Ÿชต Logger


  • @TF