Sitemap / Advertise




How to run the Edge Impulse ESP-EYE firmware on FireBeetle ESP32



Read Later




Read Later







In this tutorial, I will discuss modifying the Edge Impulse ESP-EYE firmware in order to run it on DFRobot FireBeetle ESP32 and FireBeetle Camera&Audio Media Board.

Since FireBeetle development boards and add-ons are relatively more budget-friendly than their counterparts on the market, I decided to share instructions on collecting data and running neural network models via FireBeetle ESP32 flashed with the modified Edge Impulse ESP-EYE firmware.

While modifying the ESP-EYE firmware, I encountered various errors and bugs due to the fact that FireBeetle products were not designed to run neural network models or be compatible with Edge Impulse. However, I managed to run the ESP-EYE firmware successfully in the end.

After following the instructions below, you can display a real-time video stream, capture images, and collect data from analog sensors on Edge Impulse. Nevertheless, the built-in I2S microphone is not working since it is not compatible with the current ESP-IDF version (4.4). Subsequently, I will explain the ESP-IDF I2S microphone bug in detail.


#️⃣ First of all, I installed the Edge Impulse CLI on my computer, as explained in this tutorial.

#️⃣ Then, I downloaded the Edge Impulse ESP-EYE firmware and its requirements by following these instructions.

#️⃣ Also, I installed the ESP IDF v4.4 from here.

#️⃣ After installing all the necessary tools and files, I started to modify the firmware.

📷 Camera (OV7725)

#️⃣ First, I added the correct pin connections for the FireBeetle camera&audio board in the ei_camera.h file.



  // FireBeetle Covers-Camera & Audio Media Board
  #define PWDN_GPIO_NUM     -1
  #define RESET_GPIO_NUM    0
  #define XCLK_GPIO_NUM     21
  #define SIOD_GPIO_NUM     26
  #define SIOC_GPIO_NUM     27
  #define Y9_GPIO_NUM       35
  #define Y8_GPIO_NUM       34
  #define Y7_GPIO_NUM       39
  #define Y6_GPIO_NUM       36
  #define Y5_GPIO_NUM       19
  #define Y4_GPIO_NUM       18
  #define Y3_GPIO_NUM       5
  #define Y2_GPIO_NUM       17
  #define VSYNC_GPIO_NUM    25
  #define HREF_GPIO_NUM     23
  #define PCLK_GPIO_NUM     22

Figure - 156.1

#️⃣ Although FireBeetle ESP32 does not have SPI memory (PSRAM), the camera&audio board shares pin connections with the default PSRAM connections in the ESP-IDF. Therefore, PSRAM allocation should be disabled to avoid errors.

#️⃣ To disable PSRAM, open the ESP-IDF PowerShell and enter this command: menuconfig

Figure - 156.2

#️⃣ When the configuration page pops up, go to Component config ➡ ESP32-specific and disable Support for external, SPI-connected RAM.

Figure - 156.3

Figure - 156.4

Figure - 156.5

#️⃣ After disabling PSRAM on the ESP-IDF, I modified the ei_camera.cpp file to save the image buffer to DRAM instead of PSRAM.

#️⃣ Also, I changed the pixel format since the OV7725 camera is not compatible with the JPEG format.

Available formats: YUV422, GRAYSCALE, or RGB565.

    //XCLK 20MHz or 10MHz for OV2640 double FPS (Experimental)
    .xclk_freq_hz = 20000000,
    .ledc_timer = LEDC_TIMER_0,
    .ledc_channel = LEDC_CHANNEL_0,

    .pixel_format = PIXFORMAT_YUV422, //YUV422,GRAYSCALE,RGB565,JPEG
    .frame_size = FRAMESIZE_QVGA,    //QQVGA-UXGA Do not use sizes above QVGA when not JPEG 

    .jpeg_quality = 20, //0-63 lower number means higher quality
    .fb_count = 1,       //if more than one, i2s runs in continuous mode. Use only with JPEG
	// No PSRAM
	.fb_location = CAMERA_FB_IN_DRAM,	

Figure - 156.6

Figure - 156.7

#️⃣ After flashing FireBeetle ESP32 with the modified files, you can display a real-time video stream and capture images (data) via the Edge Impulse CLI.

Figure - 156.8

Figure - 156.9

🎤 I2S Microphone (NAU8822)

#️⃣ First, I adjusted the NAU8822 I2S microphone pin connections and settings in the ei_microphone.cpp file according to DFRobot's official library.

  // Start listening for audio: MONO @ 8/16KHz
  i2s_config_t i2s_config = {
      .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_TX),
      .sample_rate = sampling_rate,
      .bits_per_sample = (i2s_bits_per_sample_t)32,
      .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
      .communication_format = I2S_COMM_FORMAT_I2S,
      .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
      .dma_buf_count = 8,
      .dma_buf_len = 128,
      .use_apll = false,
      .tx_desc_auto_clear = false,
      .fixed_mclk = -1,
  i2s_pin_config_t pin_config = {
        .bck_io_num   = 5,
        .ws_io_num    = 17,
        .data_out_num = 0,
        .data_in_num  = 39
  esp_err_t ret = 0;

Figure - 156.10

#️⃣ Even though the Edge Impulse CLI did not show any errors, I always obtained 0 from the microphone.

Figure - 156.11

Figure - 156.12

Figure - 156.13

#️⃣ I found out this is a well-known issue with the ESP-IDF versions later than 4.0 for some I2S microphones:

#️⃣ When I tested my code with the same pin connections on the Arduino IDE (ESP-IDF 3.x), it worked as expected without generating only 0 as the output.

#️⃣ However, unfortunately, the Edge Impulse ESP-EYE firmware is not compatible with previous ESP-IDF versions.

Figure - 156.14

#️⃣ Hence, I highly recommend utilizing the Edge Impulse Data Forwarder while collecting data with the built-in microphone (NAU8822) on the FireBeetle camera&audio board.

📡Analog Sensor

#️⃣ Plausibly, it does not require making any adjustments to collect analog sensor data with the Edge Impulse ESP-EYE firmware.

Figure - 156.15


Modified Code Files