[CI] External libraries compilation test (#8003)

* Workflow final + docs added Delete LIBRARY_TEST.md Update readme + documentation Generated External Libraries Test Results Updated docs Generated External Libraries Test Results revert last commit DOnt install arduino-core manually Update external_libraries_test.rst Generated External Libraries Test Results Align FQBNs to platform name use manula platform installation Generated External Libraries Test Results remove verbose remove - before sketch path Add libs and examples for specific chip Generated External Libraries Test Results Generated External Libraries Test Results Update README.md Generated External Libraries Test Results Update README.md Update README.md Generated External Libraries Test Results change name of GH action replace - for / Generated External Libraries Test Results Generated External Libraries Test Results Update lib.yml Update lib.yml Generated External Libraries Test Results remove types from pull_rewuest add pull to lib.yml Generated External Libraries Test Results add back deltas_report Update LIBRARY_TEST.md add verbose logs remove deltas report Generated External Libraries Test Results Change from schedule to push for testing add P-R-O-C-H-Y repo to cron job rename workflow Update LIBRARY_TEST.md Update LIBRARY_TEST.md Edit URL Generated External Libraries Test Results remove unused ymls + run job on schedule generated Add action link push only result file generated fix names fix workflow run workflow on push Add report to file action remove separator fix example paths fix library name yaml list of examples edit update universal sketches examples path fix update example paths Add ESP32Servo warnings -> all change warnings to "default" fix library name fix list add matrix to compile sketches test compile for only 1 chip remove GPIO example use edited action Add Blynk library use v1 print arduino cli version fix listing update libs list fix remove if fix typo fix path path test list fix fix folder creating add install IDE list fix fix missing run command Added V3 lib test + script for lib install fix typo add platform updated on: Lib testing v2 fix fqbn test all chips typo fix replace blynk with Adafruit Neopixel change blynk example fix yml format add action name Add report for PR Enable warnings report changed position typo test remove dbg level add log level option split command typo typo add empty space change position of cli flags fix typo cli compile flags not marking warning as error Add new lib+example install core manually Install core by compile-sketch fix url->path fix typo Install core from repo remove external lib checkout add lib example update board change platform name change platform name use BoardManager update example update example use repo as platform Initial commit Update README.md Update README.md * fix readme links * added JSON + lib_v2 YAML * Update lib_v2.yml * Update lib_v2.yml * Update lib_v2.yml * Update lib_v2.yml * Update lib_v2.yml * Update lib_v2.yml * Update lib_v2.yml * update JSON file * Added json file * Update lib.yml * Update lib_v2.yml * Update lib_v2.yml * change JSON * edit json + add version * edit json * update version * fix path * update fqbn platform name * add empty test result file * Add latest changes * Update docs * Update docs + clean YAML fifle * Add ESPAsyncWebServer library * Add test multiple required libs (will revert) * Update docs + json * Update lib.yml * Fix typos in docs * fix readme links
2024-09-21 10:28:04 +00:00 · 2023-04-04 11:54:42 +02:00 · 2023-04-04 11:54:42 +02:00 · df75f38a5c
commit df75f38a5c
parent 7fd956db23
11 changed files with 324 additions and 0 deletions
--- a/.github/workflows/lib.json
+++ b/.github/workflows/lib.json
@ -0,0 +1,56 @@
+[
+    {
+        "name": "Adafruit NeoPixel",
+        "version": "1.10.6",
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/Adafruit_NeoPixel/examples/strandtest/strandtest.ino"
+        ]
+    },
+    {
+        "name": "ArduinoBLE",
+        "exclude_targets": [
+            "esp32s2"
+        ],
+        "sketch_path": [
+            "~/Arduino/libraries/ArduinoBLE/examples/Central/Scan/Scan.ino"
+        ]
+    },
+    {
+        "name": "ESP32Servo",
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/ESP32Servo/examples/Knob/Knob.ino",
+            "~/Arduino/libraries/ESP32Servo/examples/Sweep/Sweep.ino",
+            "~/Arduino/libraries/ESP32Servo/examples/PWMExample/PWMExample.ino",
+            "~/Arduino/libraries/ESP32Servo/examples/Multiple-Servo-Example-Arduino/Multiple-Servo-Example-Arduino.ino"
+        ]
+    },
+    {
+        "source-url": "https://github.com/me-no-dev/ESPAsyncWebServer.git",
+        "required-libs": [
+            {"source-url": "https://github.com/me-no-dev/AsyncTCP.git"}
+        ],
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/CaptivePortal/CaptivePortal.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/ESP_AsyncFSBrowser/ESP_AsyncFSBrowser.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/regex_patterns/regex_patterns.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/simple_server/simple_server.ino"
+        ]
+    },
+    {
+        "name": "FastLED",
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/FastLED/examples/Blink/Blink.ino"
+        ]
+    },
+    {
+        "name": "IRremote",
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/IRremote/examples/SendDemo/SendDemo.ino"
+        ]
+    }
+]
--- a/.github/workflows/lib.yml
+++ b/.github/workflows/lib.yml
@ -0,0 +1,128 @@
+name: External Libraries Test
+
+# The workflow will run on schedule and labeled pull requests
+on:
+  pull_request:
+    types: [opened, reopened, synchronize, labeled]
+
+  # Schedule weekly builds on every Sunday at 4 am
+  schedule:
+    - cron: '0 4 * * SUN' 
+
+env:
+  # It's convenient to set variables for values used multiple times in the workflow
+  SKETCHES_REPORTS_PATH: libraries-report
+  SKETCHES_REPORTS_ARTIFACT_NAME: libraries-report
+  RESULT_LIBRARY_TEST_FILE: LIBRARIES_TEST.md
+  JSON_LIBRARY_LIST_FILE: .github/workflows/lib.json
+jobs:
+  compile-sketch:
+    if: |
+      contains(github.event.pull_request.labels.*.name, 'lib_test') ||
+      (github.event_name == 'schedule' && github.repository == 'espressif/arduino-esp32')
+    runs-on: ubuntu-latest
+
+    env:
+      REPOSITORY: |
+        - source-path: '.'
+          name: "espressif:esp32"
+
+    strategy:
+      matrix:
+        target:
+          - esp32
+          - esp32s2
+          - esp32c3
+          - esp32s3
+
+        include:
+          - target: esp32
+            fqbn: espressif:esp32:esp32
+          - target: esp32s2
+            fqbn: espressif:esp32:esp32s2
+          - target: esp32c3
+            fqbn: espressif:esp32:esp32c3
+          - target: esp32s3
+            fqbn: espressif:esp32:esp32s3
+
+
+    steps:
+      # This step makes the contents of the repository available to the workflow
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Compile sketch
+        uses: P-R-O-C-H-Y/compile-sketches@main
+        with:
+          platforms: |
+            ${{ env.REPOSITORY }}
+          target: ${{ matrix.target }}
+          fqbn: ${{ matrix.fqbn }}
+          use-json-file: true
+          json-path: ${{ env.JSON_LIBRARY_LIST_FILE }}
+          enable-deltas-report: true
+          sketches-report-path: ${{ env.SKETCHES_REPORTS_PATH }}
+          enable-warnings-report: true
+          cli-compile-flags: |
+            - --warnings="all"
+
+      - name: Upload artifact
+        uses: actions/upload-artifact@v3
+        with:
+          name: ${{ env.SKETCHES_REPORTS_ARTIFACT_NAME }}
+          path: ${{ env.SKETCHES_REPORTS_PATH }}
+
+  report-comment:
+    needs: compile-sketch  # Wait for the compile job to finish to get the data for the report
+    if: github.event_name == 'pull_request' # Only run the job when the workflow is triggered by a pull request
+    runs-on: ubuntu-latest
+    steps:
+      # This step is needed to get the size data produced by the compile jobs
+      - name: Download sketches reports artifact
+        uses: actions/download-artifact@v3
+        with:
+          name: ${{ env.SKETCHES_REPORTS_ARTIFACT_NAME }}
+          path: ${{ env.SKETCHES_REPORTS_PATH }}
+
+      - name: Report results
+        uses: P-R-O-C-H-Y/report-size-deltas@main
+        with:
+          sketches-reports-source: ${{ env.SKETCHES_REPORTS_PATH }}
+
+  report-to-file:
+    needs: compile-sketch  # Wait for the compile job to finish to get the data for the report
+    if: github.event_name == 'schedule' # Only run the job when the workflow is triggered by a schedule
+    runs-on: ubuntu-latest
+    steps:
+      # Check out repository
+      - name: Checkout repository    
+        uses: actions/checkout@v3
+
+      # This step is needed to get the size data produced by the compile jobs
+      - name: Download sketches reports artifact
+        uses: actions/download-artifact@v3
+        with:
+          name: ${{ env.SKETCHES_REPORTS_ARTIFACT_NAME }}
+          path: ${{ env.SKETCHES_REPORTS_PATH }}
+
+      - name: Report results
+        uses: P-R-O-C-H-Y/report-size-deltas@main
+        with:
+          sketches-reports-source: ${{ env.SKETCHES_REPORTS_PATH }}
+          destination-file: ${{ env.RESULT_LIBRARY_TEST_FILE }}
+
+      - name: Append file with action URL
+        uses: DamianReeves/write-file-action@master
+        with:
+          path: ${{ env.RESULT_LIBRARY_TEST_FILE }}
+          contents: |
+            / [GitHub Action Link](https://github.com/${{github.repository}}/actions/runs/${{github.run_id}})
+          write-mode: append
+
+      - name: Push to github repo
+        run: |
+          git config user.name github-actions
+          git config user.email github-actions@github.com
+          git add ${{ env.RESULT_LIBRARY_TEST_FILE }}
+          git commit -m "Generated External Libraries Test Results"
+          git push
--- a/LIBRARIES_TEST.md
+++ b/LIBRARIES_TEST.md
@ -0,0 +1 @@
+Empty file
--- a/README.md
+++ b/README.md
@ -1,6 +1,7 @@
 # Arduino core for the ESP32, ESP32-S2, ESP32-S3 and ESP32-C3

 ![Build Status](https://github.com/espressif/arduino-esp32/workflows/ESP32%20Arduino%20CI/badge.svg) [![Documentation Status](https://readthedocs.com/projects/espressif-arduino-esp32/badge/?version=latest)](https://docs.espressif.com/projects/arduino-esp32/en/latest/?badge=latest)
+[![External Libraries Test](https://github.com/espressif/arduino-esp32/actions/workflows/lib.yml/badge.svg?branch=master&event=schedule)](https://github.com/espressif/arduino-esp32/actions/workflows/lib.yml?link=http://https://github.com/espressif/arduino-esp32/blob/master/LIBRARIES_TEST.md)

 ### Need help or have a question? Join the chat at [Gitter](https://gitter.im/espressif/arduino-esp32) or [open a new Discussion](https://github.com/espressif/arduino-esp32/discussions)

@ -51,6 +52,11 @@ Before reporting an issue, make sure you've searched for similar one that was al

 Finally, if you are sure no one else had the issue, follow the **Issue template** or **Feature request template** while reporting any [new Issue](https://github.com/espressif/arduino-esp32/issues/new/choose).

+### External libraries compilation test
+
+We have set-up CI testing for external libraries for ESP32 Arduino core. You can check test results in the file [LIBRARIES_TEST](https://github.com/espressif/arduino-esp32/blob/master/LIBRARIES_TEST.md). 
+For more information and how to add your library to the test see [external library testing](https://docs.espressif.com/projects/arduino-esp32/en/latest/external_library_test.html) in the documentation.
+
 ### Contributing

 We welcome contributions to the Arduino ESP32 project!
--- a/docs/source/_static/cross.png
+++ b/docs/source/_static/cross.png
--- a/docs/source/_static/external_library_test_pr.png
+++ b/docs/source/_static/external_library_test_pr.png
--- a/docs/source/_static/external_library_test_schedule.png
+++ b/docs/source/_static/external_library_test_schedule.png
--- a/docs/source/_static/green_checkmark.png
+++ b/docs/source/_static/green_checkmark.png
--- a/docs/source/_static/warning.png
+++ b/docs/source/_static/warning.png
--- a/docs/source/external_libraries_test.rst
+++ b/docs/source/external_libraries_test.rst
@ -0,0 +1,132 @@
+##########################
+External Libraries Testing
+##########################
+
+About
+-----
+
+External libraries testing is a compilation test for listed libraries on arduino-esp32 master branch. The test was made for users, so they can check which libraries are compiling without errors on which SoC.
+External libraries test is running periodically (once a week) against master branch and can also run on PR by adding a label ``lib_test``.
+The test is running on all supported ESP32 chips.
+
+.. note:: 
+  As the test is just a compilation of example, that does not guarantee that the library/sketch will run without any problems after flashing it on your device.
+
+How to Add Library to Test
+--------------------------
+
+To add a library to the CI test you need to add your library to the `lib.json`_. file located in ``./github/workflows/``.
+
+.. note:: 
+  Please add the libraries to the `lib.json`_ in alphabetical order, thanks.
+
+List of parameters:
+*******************
+
+Where the library will be installed from (use only 1 option):
+
+* ``name`` - Name of the Library in Arduino Library Manager.
+* ``source-url`` - URL to the library github repository (example: "https://github.com/Arduino-IRremote/Arduino-IRremote.git"). Use when your Library is not listed in Arduino Library Manager.
+
+Required:
+
+* ``exclude_targets`` - List of targets to be excluded from testing. Use only when the SoC dont support used peripheral.
+* ``sketch_path`` - Path / paths to the sketch / sketches to be tested.
+  
+Optional:
+
+* ``version`` - Version of the library.
+* ``required-libs`` - Libraries that are necessary to be installed to compile the tested library.
+* ``destination-name`` - Folder name used for the installation of library (use only when needed).
+
+
+Example of library addition from Arduino Library Manager with ESP32-S2 excluded:
+********************************************************************************
+
+  .. code-block:: json
+
+    {
+        "name": "ArduinoBLE",
+        "exclude_targets": [
+            "esp32s2"
+        ],
+        "sketch_path": [
+            "~/Arduino/libraries/ArduinoBLE/examples/Central/Scan/Scan.ino"
+        ]
+    }
+
+Example of library addition from Github URL with required library:
+******************************************************************
+
+  .. code-block:: json
+
+    {
+        "source-url": "https://github.com/me-no-dev/ESPAsyncWebServer.git",
+        "required-libs": [
+            {"source-url": "https://github.com/me-no-dev/AsyncTCP.git"}
+        ],
+        "exclude_targets": [],
+        "sketch_path": [
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/CaptivePortal/CaptivePortal.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/ESP_AsyncFSBrowser/ESP_AsyncFSBrowser.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/regex_patterns/regex_patterns.ino",
+            "~/Arduino/libraries/ESPAsyncWebServer/examples/simple_server/simple_server.ino"
+        ]
+    },
+
+Submit a PR
+***********
+
+* Open a PR with the changes and someone from Espressif team will add a label ``lib_test`` to the PR and CI will run the test to check, if the addition is fine and the library/example is compiling.
+
+* After merging your PR, the next scheduled test will test your library and add the results to the `LIBRARIES_TEST.md`_.
+  
+Test Results
+------------
+
+Icons meaning
+*************
+
+There is a number before each icon, which means the number of examples which were compiled with that result.
+
+* |success| - Compilation was successful.
+
+* |warning| - Compilation was successful, but some warnings occurs.
+
+* |fail| - Compilation failed.
+
+* ``N/A`` - Not tested (target is in exclude_targets list).
+
+Scheduled test result
+*********************
+
+You can check the results in `LIBRARIES_TEST.md`_.
+
+The results file example:
+
+.. image:: _static/external_library_test_schedule.png
+  :width: 600
+
+Pull Request test result
+************************
+
+If the test run on Pull Request, it will compile all libraries and sketches 2 times (before/after changes in PR) to see, if the PR is breaking/fixing libraries.
+In the table the results are in order ``BEFORE -> AFTER``.
+
+.. image:: _static/external_library_test_pr.png
+  :width: 600
+
+.. |success| image:: _static/green_checkmark.png
+   :height: 2ex
+   :class: no-scaled-link
+
+.. |warning| image:: _static/warning.png
+   :height: 2ex
+   :class: no-scaled-link
+
+.. |fail| image:: _static/cross.png
+   :height: 2ex
+   :class: no-scaled-link
+
+.. _LIBRARIES_TEST.md: https://github.com/espressif/arduino-esp32/LIBRARIES_TEST.md
+.. _lib.json: https://github.com/espressif/arduino-esp32/.github/workflow/lib.json
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@ -19,3 +19,4 @@ Here you will find all the relevant information about the project.
   FAQ <faq>
   Troubleshooting <troubleshooting>
   Contributing <contributing>
+   External Libraries Testing <external_libraries_test>