QtCreator, Android Studio, Emulators, UDP To Local Network

Posted On April 7, 2022

Android studio is a very sensitive IDE. If you frown while working with it, it breaks. Qt Creator is also sensitive, and the Qt Company’s internal divisions and constant direction changes are making that platform a bit dicey as of late.

Google now has massively changed its requirements for an app to be in their store. For instance, you can’t use an older SDK, even though many older devices won’t work with the newer SDK. (There is a limit as to how far back you can go with their app compatibility layer.)

So, after about 2 years away from Android, I decided to bring up their latest Android Studio and reacquaint myself with it in order to refresh an application, and dive deeper into NDK if the Java UDP client was still broken. (SDK 26 was good, SDK 27 was broken) After that long of at time away, I expected some headache. It was almost a total disaster.

TLDR; (Too Long Didn’t Read) Results Is This:

Android Results: I had to blow away my *entire* Android SDK and NDK environment and re-install it using the new Android Studio. (Many gigabytes of data over the internet.) And then I had to Start Over Again From Scratch (SOAFS). I *tried* bringing in the older project, and letting Android Studio upgrade it, but the Android Ecosystem puked, green, blue, and fuchsia. Ugh.

Qt: Experimented with various dot revisions and Qt Creator versions to settle on one that works with Windows, Mac, and Linux.

Scroll down to TLDR; UDP… to get the solution to sending UDP packets to Android Emulator.

The Pain

While doing this work with Android and Qt (which I hoped would let me use Qt for Android Development), I found moving from one dot version of Qt to another broke everything. Nope, not a major number revision. A minor revision. In addition, the Qt Company developers changed the way they install Qt sub versions, which broke more things.

And, certain versions of Qt Creator cannot read all of the Qt project pieces properly when being created on different OS’s. I have to jump around using different versions of Qt Creator to successfully follow created “slots” for buttons between OS’s, believe it or not. (Under Qt, a slot is an automatically generated link that ties a mouse click over a button image to your code. An XML file and what I call a GUI compiler links it all together.) And the version direction for working Qt Creators between OS’s is not always forward!

My Linux Creator 4.8 reads all the slots ok. Mac Creator 4.15 and 6.02 fails to read all the slots. However, on the same machine, Creator 5.03 reads the slots just fine. Creator 7.0.0 sometimes just abruptly quits when trying to “go to” a slot. Mac is my main development platform. (Don’t get me started ranting on Windows!)

Broken Open Source Conundrum

Yes, I understand that it is *all* open source and I can spend countless hours fixing the source code. Sorry, I am not 20 years old, hyped on caffeine, with no girlfriend, with this being brand new interesting stuff. Nope, not interesting. Not brand new.

I ported MSDOS to an RTOS kernel, chasing BIOS cassette timing routines. I ported CP/M 68K to the Mac. I have deployed a critical database over an entire state. I created a system that converted images brought in from around the world with tight deadlines so people got money properly. I create electronics that run at 300+ degrees F.

After trivial things like that, this is no longer fun, interesting, or entertaining. I have employees whose families need feeding, I can’t spin wheels.

Mobile Device Info

There are around 2 billion mobile devices, and only about 5% to 10% of them are IOS devices. In addition, Apple plays dirty, and without a hefty law firm you can’t swim in their pool fairly. Google is approaching that level of insanity, but their stuff is open source and there are other ways to service Android without Google Play Store.

The average lifetime earnings of an IOS application at the beginning was $3,000. I spent around $10,000 creating an app, and it has earned about 1/2 of that money back before Apple’s hostility to developers and their constantly changing goal posts killed it. So, Apple Mobile is firmly gone from my list. No more in that arena. Their IDE is horrific now. You spend 5% of your time on app, 95% of your time fighting the IDE and Apple. I have a life to lead, apart from Apple.

Just to be fair, Android application earning potentials are much less. Their goal posts are also always changing, meaning you have to pump money and time in for an app that updates for free. There are always winners, but not every application is an angry avian app. Yours most likely will not be the next one. My results for Android are similar to Apple, but Google has been less evil in the past. (They have officially removed that from their goals, so they can be evil now.)

So, my experience with so much investment for so little return has relegated all the mobile platforms to the back burner. No, I am not interested in the latest and greatest, there is no way to make a living supporting constantly moving goalposts. I can’t even earn minimum wage selling apps on those platforms, and according to the platform statistics, my apps are good performers!

The *only* way to justify mobile platform development is in conjunction with hardware. And that leads to the next bit, a hardware piece sending info to an emulated mobile device.

One thing to note, I was unable to find an answer for my problem anywhere as a single posted solution. My answer is boiled down to a single post solution from an amalgam of various posts and much experimentation.

Mobile Devices and PSOC: Introduction

My closed project with an embedded PSOC5 generates UDP packets over a network, using Qt to do this. (Using LGPL linking with Qt Libraries. Other projects are open source, also using LGPL linking.) The data is picked up by various platforms, including Android.

So, I needed to send the packets to the Android Emulator. Easy, Peasy. Not! Read On For The Solution.

Android Emulator And Networking

With Android Studio, the emulator is in a network sandbox. You can’t easily get to the Emulated Android device network from your pc. It is even harder to get to the device from a network . That makes testing device to emulated mobile device or PC to emulated mobile device very difficult. When you run a Mac with VM’s it is even more convoluted. You can have a dozen or more networks running around just on your machine.

TLDR; UDP Network Packets To Your Android Device

This is from a Mac point of view. My UDP port is 1234. Change it to your port. Here is what I did:

1) created file ".emulator_console_auth_token" in home dir, with myusername in it as the only text in that file
2) telnet localhost 5554
3) auth myusername
4) redir add udp:1234:1234
5) quit
6) nc -k -w 0  -l -u 0.0.0.0 1234 | nc -u 127.0.0.1 1234

That works. However, you have to do most of that every time. And repeats, without terminating the emulator, force you to unlink the redir and re-establish it. I finally created a way to more or less automate it.

Setup

First, make sure telnet is installed on your machine. Second, make sure that netcat is installed on your machine. Third, make sure “expect” is installed on your machine. Those steps are left as an exercise for the reader. (Favorite Professor Saying.)

Fourth, on Mac (and possibly Linux), create two scripts, make them executable. The first script uses expect. I call it forward1234.sh, because I am port forwarding UDP port 1234. Here is its contents:

#!/usr/bin/expect
spawn telnet localhost 5554
set timeout 10
expect "OK"
send "auth myusername\n"
expect "OK"
send "redir del udp:1234\nredir add udp:1234:1234\nquit\n"
expect "Connection closed by foreign host."

Then I created fw.sh, because I wanted a simple filename:

#!/bin/bash
./forward1234.sh
echo "running netcat forwarder.  ^C to quit......"
/usr/bin/nc -k -w 0  -l -u 0.0.0.0 1234 | /usr/bin/nc -u 127.0.0.1 1234

I made 2 scripts because the expect script (in simplified form) cannot handle spawning, quitting, and respawning. I am sure that if I studied “expect” program I could overcome that, but I don’t have time, and this is working.

So, I bring up a terminal, and run ./fw.sh (with forwared1234.sh being in the same directory) and the link is established. My network packets to that port show up at the emulator, and I can debug and test. I go to the terminal and hit Ctrl-C to abort the setup.

Things To Consider About expect And netcat

Expect being used in this mode is brain dead. But it works. The “redir del” is for past invocations that have been left in place. The \n allows you to stack commands on one line.

Netcat wants to only forward 1 packet. Period. However, if you combine the -k command line option (keep running (respawning) even if it wants to quit) along with the -w 0 command line option (wait 0 seconds for a packet before quitting) you have a continuously restarting packet forwarder.

This continuous restart is slow. It misses about 80% of the UDP packets. But enough get through to work. And this brings up a comment about TCP and UDP.

UDP Is Superior To TCP For Sensor Data Over Damaged Links

TCP is broken in regards to mobile networks. With a packet loss of 30% or more, TCP will hang. Hard. It can be handled, but none of the stacks that I have seen handle that much packet loss. How many web pages have you seen to be exceedingly slow to load (or time out) because your signal was not so good?

UDP, on the other hand, doesn’t care. You will get the packets. If you handle your information flow properly in your code, you can survive 90% packet loss and still get enough sensor measurements through to do something useful (although maybe not in absolute real time).

Human Cognition Rules (May Surprise You)

IBM did some HID studies before the middle 1990’s. They found the average person takes about two seconds to look at numbers or letters on a screen and turn those items into understanding, making a decision, performing an action.

During those two seconds, no subjective time passes for the person. That person thinks the information was immediately handled. However, a “zone out” occurred. But two seconds really elapsed. Even so, you will get argued with if you declare the time passage.

Therefore, if during that two seconds, you generate 10 UDP packets on the network with a 90% packet loss, at least one packet will make it through. That packet will have the latest information. And the information will be “real time” for the human. Effective decisions can still be made. Interesting.

Bad Connection TCP Problems

With TCP, you will get bad information, as the handshaking code is still trying to get the first packet to come through. Eventually, if TCP does not give up, your data will be two to ten seconds (or more) old before information arrives. So, the displayed information is two to ten seconds (or more) out of date.

That is one of the major problems with MQTT. It demands TCP. (There is at least one version of MQTT using UDP for that very reason.) My son, who is a skilled engineer, had to patch an implementation of TCP to fix this very problem. TCP will fail on a network with 30% packet loss. I.E. wireless.

Therefore, UDP is the best transport for receiving data from sensory devices like a PSOC driven device. Low overhead on the transmitter (just shove on the wire and forget about it). Low overhead on the receiver (just handle a packet as it arrives). It is a little tougher on a network, as packets usually are broadcast to every device, unless you do some initial handshaking to limit the destination(s).

UDP Rules Of Thumb

Keep your broadcasts for UDP to your local network using a proper netmask. If you have a DHCP router, use the netmask provided by DHCP to limit your UDP broadcasts. Otherwise, you can almost always safely go with a .255 netmask, as almost all networks are NAT now, with around a 254 machine limit. Making broadcast network information settable by the user is useful under certain situations.

My PSOC driven device generates packets for UDP broadcast around 4 times per second. (The PSOC is not exactly loafing, but it could generate packets much faster.) That allows the netcat to provide about 2 packets per second to the android emulator, and it keeps the current information up to date.

However, even on an 8 core machine, running an emulator to process data packets for Android is slow and painful. It is good enough for debugging, and magically covers situations where packets are lost. Very useful for the real world.

On to the next set of artificially generated problems.

Enjoy!