LLM code assistance

LLM code assistance#

We will setup LLM code assistance in our editor so that the LLM can interact with our code directly. You are probably already using an LLM agent and installing LLM support in our editor will ease the interaction and save you copy & paste time.

For using LLMs directly in our editor, we have two options:

  1. Running a local LLM on our computer

  2. Using cloud services from companies like Mistral, OpenAI, Anthropic, etc.

Without a powerful graphics card or neural-network accelerator, the first option will probably be very slow. On the other hand most cloud services are not free to use including OpenAI. Even ChatGPT is free to use, using it as a remote cloud application, i.e., using their API (application programming interface), is not free.

As of the writing of this, Groq has a free tier, which allows about thousand requests per day for the model meta-llama/llama-4-scout-17b-16e-instruct, which should be sufficient for an educational setting. If you run out of credits, you can always use a free to use model through their web interface, e.g., Mistral, ChatGPT, etc.

Let us first get an API key from Groq and then use it in the Code LLM extension Continue.

Creating a Groq API key#

  1. Go to https://console.groq.com.

  2. Create an account. You will be logged in to the dashboard after creating an account.

  3. Click on Create API key. You will immediately be shown an API key.

  4. Copy the API key.

Installation of Continue#

  1. In Code, install the extension Continue - open-source AI code assistant. It may take about 1 minute.

    After the installation, you will see a settings icon right to the Auto Update on the tab of the extension and also on to the Continue app in the extension listing.

    You will also notice the new Continue icon on the activity bar.

  2. Click on one of the s. A menu will pop up.

  3. Click on Settings. Settings tab will open up and filter for the settings of Continue.

  4. (optional) Go to Telemetry Enabled and opt out telemetry if you desire.

  5. Close Settings.

  6. Move the Continue icon from the first step to the secondary sidebar on the right as shown here.

  7. Click on Or, configure your own models. A configuration window will open up.

  8. Right below the Connect button, click on Click here to view more providers. Add Chat Model window will pop up.

  9. Select the provider Groq.

  10. Select in the model drop-down menu Autodetect.

  11. Paste the API key that you have here into the corresponding field.

  12. Click Connect. The tutorial file continue_tutorial.py and the text-based configuration file config.yaml will open up in new tabs. In the latter file you can configure Models for different purposes, e.g., Chat, Autocomplete, etc.

    You can close these two files.

  13. For Chat, Edit, and Apply, choose the model that contains llama-4-scout in its name.

    We are not going to setup Autocomplete and Rerank.

  14. Click on the on the left side of Models to collapse the toolbar.

Usage examples: explanation and formatting#

  1. Close continue_tutorial.py tab, we are going to try the agent on the pi-estimator code.

  2. On the Continue tab, if the a subwindow called Create Your Own Assistant is open, you can close it.

  3. Click on the tab with the pi-estimator code.

  4. Write on the prompt explain me the code line by line and press AltEnter. Without Alt, Continue does not send your code to the agent.

    You should see an explanation.

  5. Now write format code. LLM should reply with formatted code. I got the following:

    #include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main() {
    int total = 1000000;
    int inside = 0;

    srand(time(0));

    for (int i = 0; i < total; i++) {
        double x = (double)rand() / RAND_MAX;
        double y = (double)rand() / RAND_MAX;

        if (x * x + y * y <= 1)
            inside++;
    }

    double pi = 4.0 * inside / total;
    printf("Estimated pi: %f\n", pi);

    return 0;
}

    Even I prompted format code, the model added also return 0; line, which is not required, but can be part of an explicit code style. When I tried another model, llama-4-maverick, it did not include return 0. So pay attention that you review the changes block by block and ask different models in doubt.

    You notice that formatting looks different than the language server we used. LLM is able to recognize blocks that logically belong together and insert newlines accordingly, which is more advanced compared to the clangd formatter we used. Nevertheless, clangd formatter should be just fine for daily work. I just used this example to showcase an LLM’s difference to a formatter.

  6. To apply the changes: On the right top corner of the code reply, click on Apply. Accept | Reject blocks will appear in your code.

    As recommended before, accept or reject block by block – especially if you are in the process of getting to know a model.

Tip

One of the learning goals of this course is to identify and explain core programming concepts without an LLM to be able to criticize the output of a LLM. I recommend you to write the programs yourself and ask the LLM as a second step only for getting feedback, improvements or explanations.

You probably already know that ChatGPT can keep a memory of your preferences and you can customize ChatGPT with custom instructions like prefer being direct over being nice when giving feedback. You can also customize your code assistant’s traits. How see how to do that in Continue, click here for details.

Contents