Remote Robot Arm, properly

Being able to control the Robot arm using a VNC client to connect to the Raspberry Pi is all very good but it's not a very elegant form of remote control. So, I decided to build an Android App on top of the work to date.

The App should have the following behaviour:

1. It should be able to display a stream from a web cam attached to the Raspberry PI.
2. Control of the robot would be by pressing down on buttons on the screen. Releasing the button should stop the robot.
3. At this point I only need to control this over the local network.

Remote Connection

Requirement 3 meant a simple TCP Client-Server model would suit me well.

As this is only on the local network, I have control over the IP address assigned by the router. I hard-coded the host name and port on both Server and client.

On the robot-arm server it's only a matter of adding a few lines to enable TCP

   import socket

   TCP_IP = "192.168.0.12"
   TCP_PORT = 5005
   BUFFER_SIZE = 20 # Normally 1024, but we want a fast response

   s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   s.bind((TCP_IP, TCP_PORT))
   s.listen(1)

   conn, addr = s.accept()

and in the loop

   while 1:
      data = conn.recv(BUFFER_SIZE)
      if not data: break
      # send the chractera to the arduinos


On the client side I created a new java.net.Socket and use a PrintWriter to send the control characters:

   Socket socket = new Socket(hostName, port);
   PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
   out.println(ch);


Note that as this is an Android App, it's mandated that sockets are not opend on the main thread. It is very simple to extend AsyncTask and implement the doInBackground method.

Finally, we need to add Internet permission to the Android manifest file:

<uses-permission android:name="android.permission.INTERNET" />

Streaming a Video feed

Keeping things simple I thought an HTTP stream would be lightweight and easy to implement. It turns out most solutions described on the web introduce a large latency of several seconds.
Fortunately however, I found a solution in the PiStreaming project on GitHub https://github.com/waveform80/pistreaming.git 

Following the install instructions and starting with python server.py I was able to view the stream from the camera with sub-second latency.

Building the App

The app has two regions, the top half shows the video stream while the bottom has the buttons to control motion

I use a WebView to show the video stream. In OnCreate it's configured with the following:

   WebView piWebView = (WebView)findViewById(R.id.webview);
   WebSettings webSettings = piWebView.getSettings();
   webSettings.setJavaScriptEnabled(true);
   piWebView.setWebViewClient(new WebViewClient()
   {
      @Override
      public boolean shouldOverrideUrlLoading(WebView view, String url)  
      {
         return false;
      }
   }); 

   piWebView.loadUrl("http://192.168.0.12:8082");

Java script must be enabled because the HTTP server uses it and the override shouldOverrideUrlLoading was added to stop the stream spawning a new web browser window.

To implement the required behaviour of the buttons, I used the OnTouchListener. The MotionEvents ACTION_DOWN and ACTION_UP capture the button press and release

m_upButton.setOnTouchListener(new OnTouchListener(){
   @Override
   public boolean onTouch(View v, MotionEvent event) {
    try
    {
     if (event.getAction() == MotionEvent.ACTION_DOWN)
     {
      new TcpClient().execute("o");
     }
     else if (event.getAction() == MotionEvent.ACTION_UP)
     {
      new TcpClient().execute(" ");
     }
    }
    catch(Exception ex)
    {
     return false;
    }
  
    return true;
   }
  });

There are a couple of problems here. I shouldn't need to create the TCP client each time and there is an intermittent but where the ACTION_UP pipeline doesn't fully make it to the robot, in this case the arm continues moving until I tap another button.

Running the App to control the robot

On the Raspberry PI start both the web server and the robo-server. Then open the Robot Arm App. Touching the buttons cause the appropriate motors on the arm to move, releasing the button stops movement.

The video shows a demo of it in action. I'm using a low spec Android phone and hadn't connected all the motors but it gives an impression of how this works

The code for the Android App has been added to the git repository https://github.com/jscott7/Robot-Arm.git