Github -> raia Agent

As a user of Raia Agent, I want to automatically sync the code from a GitHub repository into a Raia Agent as a Markdown file. This will provide the agent with a comprehensive understanding of our codebase, enabling it to answer technical questions, assist with onboarding new developers, and provide context-aware support.

The n8n workflow should run on a nightly schedule and perform the following actions:

1. Fetch Repository Contents: Recursively fetch the entire file structure and content of a specified GitHub repository.

2. Consolidate to Markdown: Convert the fetched code into a single, well-structured Markdown file. The file should include:

   • Repository Name and Description.

   • A file tree or table of contents.

   • The content of each relevant file, properly formatted in code blocks with syntax highlighting.

   • Clear metadata, including the file path for each code block.

3. Upload to Raia Agent: Upload the generated Markdown file to the Raia Agent using the API.

Acceptance Criteria:

• An n8n workflow is created that successfully fetches all relevant files from a specified GitHub repository.

• The workflow consolidates the code into a single Markdown file with the specified structure and metadata.

• The workflow successfully uploads the Markdown file to the Raia Agent via the API.

• The workflow is scheduled to run every night.

• The workflow includes robust error handling and notifications for failed runs.

Technical Details:

• GitHub API:

  • Authentication: Bearer Token (Authorization: Bearer YOUR_GITHUB_TOKEN).

  • Recommended Approach: Use the Git Trees API for efficient recursive fetching.

  • Key Endpoints:

    • GET /repos/{owner}/{repo}/git/trees/{tree_sha}?recursive=1: To get the entire repository file tree in a single call.

    • GET /repos/{owner}/{repo}/contents/{path}: To get the content of individual files (which will be Base64 encoded).

• Raia Agent API:

  • Endpoint: POST /external/agent-files/upload

  • Request Type: multipart/form-data

  • Authentication: API Key in the header.

  • Request Body:

    • file: The Markdown file to upload.

    • autoAddToVectorStore: true (to make the content searchable by the agent).

Implementation Notes:

• The n8n workflow will require credentials for both the GitHub API and the Raia Agent API.

• The workflow should be optimized to handle large repositories by using the Git Trees API.

• Implement a filtering mechanism to include only relevant file extensions (e.g., .py, .js, .md) and exclude binaries, images, and dependency directories (e.g., node_modules).

• The content fetched from the GitHub Contents API is Base64 encoded and will need to be decoded before being written to the Markdown file.

• Use appropriate Markdown syntax for code blocks, specifying the language for syntax highlighting (e.g., ```python).

• The workflow should be tested with a sample GitHub repository before being deployed to production.