Always receiving a BLE count of zero when using Arduino IDE

[zapoig]

Environment :

• Windows 10 (also reproduced on Linux)
• Arduino IDE 1.8.19
• ESP32 espressif arduino boards packages 1.0.6
• T-Beam v1.0 dev board (reproduced on many different other boards)

Steps to reproduce :

• clone libpax branch master
• edit lib/libpax/libpax_api.h to uncomment build-time options LIBPAX_WIFI and LIBPAX_BLE
• copy lib/libpax in arduino libraries folder
• copy the sample program src/main.cpp as src/main.ino and open it in arduino ide
• modify it like this (removing non-arduino version for clarity, and defining LIBPAX_ARDUINO as instructed in README.md) :

#define LIBPAX_ARDUINO
#include <libpax_api.h>
#include <stdio.h>

struct count_payload_t count_from_libpax;

void process_count(void) {
  printf("pax: %d; %d; %d;\n", count_from_libpax.pax, count_from_libpax.wifi_count, count_from_libpax.ble_count);
}

void init() {
  struct libpax_config_t configuration; 
  libpax_default_config(&configuration);
  configuration.blecounter = 1;
  configuration.blescantime = 0; // infinit
  configuration.wificounter = 1; 
  configuration.wifi_channel_map = WIFI_CHANNEL_ALL;
  configuration.wifi_channel_switch_interval = 50;
  configuration.wifi_rssi_threshold = -80;
  configuration.ble_rssi_threshold = -80;
  libpax_update_config(&configuration);

  // internal processing initialization
  libpax_counter_init(process_count, &count_from_libpax, 10, 1); 
  libpax_counter_start();
}

void setup() {
  init();
}

void loop() {}

• build and upload, open the serial monitor.

Expected output:

Receiving values for both wifi and ble counts.

Actual output:

16:25:09.268 -> pax: 3; 3; 0;
... 10 minutes later
16:25:19.262 -> pax: 7; 7; 0;
BLE count is always zero.

Note that running the BLE_scan sample sketch included with ESP32 arduino package finds around 15 BLE devices at the same location, same time on the same board.

[zapoig]

Reproduced again with the beta Arduino IDE 2.0 RC3.

Going to try with esp-idf framework, that might take me a while since I never used it before.

[cyberman54]

@zapoig could you please tun the test cases, and report results here?

pio test -e arduino

[zapoig]

Here you are :

Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in arduino environment
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
In file included from lib\libpax/libpax.h:4:0,
                 from test\libpax_test_cases.cpp:1:
lib\libpax/libpax_api.h:7:0: warning: "LIBPAX_WIFI" redefined
 #define LIBPAX_WIFI // enables WiFi sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax/libpax.h:4:0,
                 from test\libpax_test_cases.cpp:1:
lib\libpax/libpax_api.h:8:0: warning: "LIBPAX_BLE" redefined
 #define LIBPAX_BLE  // enables BLE sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax_api.cpp:19:0:
lib\libpax/libpax_api.h:7:0: warning: "LIBPAX_WIFI" redefined
 #define LIBPAX_WIFI // enables WiFi sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax_api.cpp:19:0:
lib\libpax/libpax_api.h:8:0: warning: "LIBPAX_BLE" redefined
 #define LIBPAX_BLE  // enables BLE sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\blescan.cpp:5:
lib\libpax\libpax_api.h:7:0: warning: "LIBPAX_WIFI" redefined
 #define LIBPAX_WIFI // enables WiFi sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\blescan.cpp:5:
lib\libpax\libpax_api.h:8:0: warning: "LIBPAX_BLE" redefined
 #define LIBPAX_BLE  // enables BLE sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\libpax.cpp:21:
lib\libpax\libpax_api.h:7:0: warning: "LIBPAX_WIFI" redefined
 #define LIBPAX_WIFI // enables WiFi sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\libpax.cpp:21:
lib\libpax\libpax_api.h:8:0: warning: "LIBPAX_BLE" redefined
 #define LIBPAX_BLE  // enables BLE sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\wifiscan.cpp:36:
lib\libpax\libpax_api.h:7:0: warning: "LIBPAX_WIFI" redefined
 #define LIBPAX_WIFI // enables WiFi sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from lib\libpax\libpax.h:4:0,
                 from lib\libpax\wifiscan.cpp:36:
lib\libpax\libpax_api.h:8:0: warning: "LIBPAX_BLE" redefined
 #define LIBPAX_BLE  // enables BLE sniffing features in build
 ^
<command-line>:0:0: note: this is the location of the previous definition
C:\Users\sesa334427\.platformio\packages\framework-arduinoespressif32\cores\esp32\esp32-hal-spi.c: In function 'spiTransferBytesNL':
C:\Users\sesa334427\.platformio\packages\framework-arduinoespressif32\cores\esp32\esp32-hal-spi.c:922:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
                 uint8_t * last_out8 = &result[c_longs-1];
                                       ^
C:\Users\sesa334427\.platformio\packages\framework-arduinoespressif32\cores\esp32\esp32-hal-spi.c:923:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
                 uint8_t * last_data8 = &last_data;
                                        ^
Uploading...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8
test\libpax_test_cases.cpp:212:test_mac_add_bytes       [PASSED]
test\libpax_test_cases.cpp:213:test_collision_add       [PASSED]
test\libpax_test_cases.cpp:214:test_counter_reset       [PASSED]
test\libpax_test_cases.cpp:215:test_config_store        [PASSED]
after start: Current free heap: 4353424
libpax should be running
pax: 13; 1; 12;
pax: 18; 1; 17;
pax: 19; 1; 18;
pax: 19; 1; 18;
pax: 20; 1; 19;
pax: 22; 1; 21;
E (7940) event: Event loop not initialized via esp_event_loop_init, but esp_event_send called
after stop: Current free heap: 4407904
libpax should be stopped
after start: Current free heap: 4407844
libpax should be running
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
after stop: Current free heap: 4407844
libpax should be stopped
after start: Current free heap: 4372956
libpax should be running
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 2; 2; 0;
pax: 2; 2; 0;
E (21961) event: Event loop not initialized via esp_event_loop_init, but esp_event_send called
after stop: Current free heap: 4407720
libpax should be stopped
after start: Current free heap: 4385940
libpax should be running
pax: 15; 0; 15;
pax: 17; 0; 17;
pax: 18; 0; 18;
pax: 19; 0; 19;
pax: 20; 0; 20;
pax: 22; 0; 22;
after stop: Current free heap: 4404856
libpax should be stopped
after start: Current free heap: 4369960
libpax should be running
pax: 0; 0; 0;
pax: 3; 3; 0;
pax: 3; 3; 0;
pax: 3; 3; 0;
pax: 3; 3; 0;
pax: 3; 3; 0;
E (35972) event: Event loop not initialized via esp_event_loop_init, but esp_event_send called
after stop: Current free heap: 4404732
libpax should be stopped
after start: Current free heap: 4346968
libpax should be running
pax: 12; 0; 12;
pax: 18; 0; 18;
pax: 19; 0; 19;
pax: 21; 0; 21;
pax: 21; 0; 21;
pax: 22; 0; 22;
E (42976) event: Event loop not initialized via esp_event_loop_init, but esp_event_send called
after stop: Current free heap: 4401796
libpax should be stopped
test\libpax_test_cases.cpp:216:test_integration [PASSED]
-----------------------
5 Tests 0 Failures 0 Ignored
================================================================================ [PASSED] Took 82.73 seconds ================================================================================

Test    Environment    Status    Duration
------  -------------  --------  ------------
*       arduino        PASSED    00:01:22.732
================================================================================ 1 succeeded in 00:01:22.732 ================================================================================

**********************************************************************************************************************************************************************************************
There are the new updates for platforms (espressif32)
Please update them via `platformio platform update` command.

If you want to manually check for the new versions without updating, please use `platformio platform update --dry-run` command.
**********************************************************************************************************************************************************************************************


**********************************************************************************************************************************************************************************************
We found 274.89MB of unnecessary PlatformIO system data (temporary files, unnecessary packages, etc.).
Use `pio system prune --dry-run` to list them or `pio system prune` to save disk space.

[zapoig]

All tests are also successful with pio test -e espidf.

[cyberman54]

Tests show BLE sniffing results (see third column) as expected, thus BLE sniffing works.

Looks like in integration / API problem with your app. Did you set the build time options in your app?

#define LIBPAX_WIFI // enables WiFi sniffing features in build 
#define LIBPAX_BLE  // enables BLE sniffing features in build

[zapoig]

At the moment I am not using any code of my own, I'm testing with only the libpax provided sample "src/main.cpp"

As described in my initial post I have uncommented the build time options directly in the libpax file libpax_api.h so they are always enabled.

[cyberman54]

In the example there is an RSSI threshold set:

configuration.wifi_rssi_threshold = -80;
configuration.ble_rssi_threshold = -80;

Perhaps your BLE signals do not hit the threshold. Please set to zero and try again.

[zapoig]

Nope it doesn't change the outcome.

Since the only time it actually detected some BLE ids was in the integration test build by platformio, my next attempt will be to try to build the sample code using pio.

Will let you know the result, thanks for your help.

[cyberman54]

I installed Arduino IDE 1.8.19, compiled and uploaded the main.cpp on a ESP32 T-Beam Board, an can reproduce this issue:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5816
entry 0x400806ac
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;
pax: 0; 0; 0;

The same board, same software, but compiled and uploaded with platformio (frameworkespressif32 v3.5.0 which is based on Arduino-espressif32 v1.0.6) does work.

[zapoig]

That's similar to my own results. Thanks for your time testing that.

[zapoig]

I finally succeeded in building the main.cpp sample with esp-idf v4.4, and the BLE counting works there.

[cyberman54]

@zapoig thanks for the note.
We should leave this open, until it's clear why it does not work with Arduino IDE.

[bleeptrack]

I'm also experimenting with libpax and ran into the same issue:
Building with platform.io and this configuration

[env:nodemcu-32s]
platform = espressif32
board = nodemcu-32s
framework = arduino

gives me 0 BLE devices.

[zapoig]

I'm also experimenting with libpax and ran into the same issue: Building with platform.io and this configuration

[env:nodemcu-32s]
platform = espressif32
board = nodemcu-32s
framework = arduino

gives me 0 BLE devices.

Did you try to run the tests ?
pio test -e arduino

[lifezoned4]

In the example there is an RSSI threshold set:

configuration.wifi_rssi_threshold = -80;
configuration.ble_rssi_threshold = -80;

Perhaps your BLE signals do not hit the threshold. Please set to zero and try again.

I can only support that this rssi threshold can lead to none or only few visible devices. Setting the threshold to default 0 should at least show something similar to the unittest.

@zapoig @bleeptrack can you confirm that this still leads to no BLE counts in a noise environment? And do unit test yield counts?

[zapoig]

@lifezoned4 Yes I had tried setting rssi_thresholds (for both wifi and ble) to zero before.
Here is the results I have today :

When unit tests yield counts :

libpax should be running
pax: 18; 1; 17;
pax: 21; 1; 20;
pax: 21; 1; 20;
pax: 21; 1; 20;
pax: 23; 1; 22;
pax: 23; 1; 22;
E (41212) event: Event loop not initialized via esp_event_loop_init, but esp_event_send called
after stop: Current free heap: 208120
libpax should be stopped

The sample, built with arduino IDE, only finds wifis with default RSSIs = -80 :

pax: 1; 1; 0;
pax: 1; 1; 0;
pax: 2; 2; 0;
pax: 2; 2; 0;
pax: 3; 3; 0;
pax: 3; 3; 0;

And - still building using arduino IDE - switching the RSSIs to 0 doesn't change the BLE zero count (although it appears to find more wifis) :

pax: 4; 4; 0;
pax: 7; 7; 0;
pax: 8; 8; 0;
pax: 11; 11; 0;
pax: 14; 14; 0;
pax: 15; 15; 0;

Now if I build the original sample (rssi at -80) with platformio, it is finding both ble and wifi nodes :

pax: 6; 1; 5;
pax: 6; 1; 5;
pax: 6; 1; 5;
pax: 6; 1; 5;
pax: 6; 1; 5;
pax: 8; 1; 7;

And switching the rssi to 0 improves both counts - with platformio build again :

pax: 16; 2; 14;
pax: 17; 2; 15;
pax: 17; 2; 15;
pax: 17; 2; 15;
pax: 20; 2; 18;
pax: 21; 2; 19;

In the end I am happy having switched to platformio for my own project, but if the arduino IDE support is broken it should probably get mentioned in the README.

[cyberman54]

@zapoig thanks for the report. It's strange, i have no idea what's the root cause here.