Journey in Rust: Prompt Tuning in ChatGPT API - Part 3

In the previous part of our journey, we learned how to make requests to the OpenAI API and handle the responses. Today, we'll continue enhancing our command line tool by:

1. Adding an optional max_tokens argument to control the length of the generated response.

2. Detecting the user's operating system and incorporating it into our query.

Let's dive in! 🏊‍♂️

Step 1: Adding the max_tokens argument

Let's begin by adding an optional max_tokens argument to our Args struct:

#[derive(Parser, Debug)]
#[clap(
    version = "0.1.0",
    author = "Your Name <your.email@example.com>",
    about = "A simple command line tool"
)]
struct Args {
    #[clap(about = "The query you want to run")]
    query: String,

    #[clap(short, long, about = "The maximum number of tokens in the response")]
    tokens: Option<u32>,
}

With this addition, users can now provide the --tokens or -t flag followed by an integer value to control the maximum number of tokens in the generated response.

Step 2: Prompt Tuning

As we worked with the command line tool and the GPT-3.5 model, we encountered different situations where the model provided responses with varying levels of detail. Let's go through a step-by-step process to understand how we reached the conclusion that providing OS information is essential for better results.

Step 1: We started with a simple system command.

• System: "Act as a terminal expert"

• User: "Convert mov to mp4 using ffmpeg"

• Result: "To convert a MOV video file to MP4 using FFmpeg, follow these steps:

  1. Download and install FFmpeg if you haven't already.

  2. Open a Command Prompt or Terminal window on your computer.

  3. Navigate to the directory where your MOV file is located using the "cd" command.

  4. Use the following command to convert the MOV file to MP4:

ffmpeg -i input.mov -c:v libx264 -preset slow -crf 22 -c:a copy output.mp4"

Detailed instructions, but too much information

Step 2: Updated the system command for a more concise answer.

• System: "Act as a terminal expert, if user is asking for command give COMMAND ONLY"

• User: "Convert mov to mp4 using ffmpeg"

• Result: ffmpeg -i input.mov -c:v libx264 -preset slow -crf 22 -c:a copy output.mp4

Perfect! Concise command without extra details

Step 3: Tested without mentioning ffmpeg explicitly.

• System: "Act as a terminal expert, if user is asking for command give COMMAND ONLY"

• User: "Convert mov to mp4"

• Result: ffmpeg -i input.mov -c:v libx264 -preset slow -crf 22 -c:a copy output.mp4

Still got the correct command!

Step 4: Tried a different type of question.

• System: "Act as a terminal expert, if user is asking for command give COMMAND ONLY"

• User: "How to check if I am connected to wifi"

• Result: "The command to check if you are connected to wifi varies depending on the operating system you are using. Here are the commands for some common operating systems:

  - Windows: Type netsh wlan show interfaces in the Command Prompt or PowerShell. - macOS: Type networksetup -getairportnetwork en0 in the Terminal. - Linux: Type iwconfig or nmcli dev wifi in the Terminal.

  Note: Replace en0 with the appropriate interface name for your macOS device, and make sure your Linux distribution has the appropriate tools installed to execute these commands."

Received commands for multiple operating systems (Windows, macOS, Linux)

Step 5: Provided operating system information in the system command.

• System: "Act as a terminal expert, answer should be the COMMAND ONLY, no need to explain. OS: Macos"

• User: "How to check if I am connected to wifi."

• Result: ifconfig en0 | grep status | awk '{print $2}

A concise command specifically for macOS!

This process showed us that providing OS information is essential for accurate and concise responses from the GPT-3.5 model. As a result, we implemented OS detection in Rust and included it in our API query for better results.

Step 3: Detecting the user's operating system

To detect the user's operating system, we'll create a new function called get_pretty_name(). This function will return a Result<String> that represents the operating system's name. Before we provide the code, let's walk through how this function will work:

1. We will use the OS constant from the env::consts module to match against the user's operating system.

2. For Linux systems, we will read the /etc/os-release file and look for a line starting with PRETTY_NAME=. If found, we will return the pretty name value as a string.

3. For Windows systems, we'll simply return the string "Windows".

4. For macOS systems, we'll return the string "macOS".

5. In case an unknown operating system is detected, we'll return an error.

Now let's look at the code implementation of get_pretty_name():

use std::{
    env::{self, consts::OS},
    fs::File,
    io::{self, BufRead},
    path::Path,
};

// ...

fn get_pretty_name() -> io::Result<String> {
    match OS {
        "linux" => {
            let os_release_path = Path::new("/etc/os-release");
            let file = File::open(&os_release_path)?;
            let reader = io::BufReader::new(file);
            for line in reader.lines() {
                let line = line?;
                if line.starts_with("PRETTY_NAME=") {
                    return Ok(line[13..line.len() - 1].to_string());
                }
            }
            Err(io::Error::new(
                io::ErrorKind::NotFound,
                "PRETTY_NAME not found in /etc/os-release",
            ))
        }
        "windows" => Ok("Windows".to_string()),
        "macos" => Ok("macOS".to_string()),
        _ => Err(io::Error::new(
            io::ErrorKind::NotFound,
            "Unknown operating system",
        )),
    }
}

This function uses the OS constant from the env::consts module to match against the user's operating system. We handle Linux, Windows, and macOS cases specifically, returning an error if an unknown operating system is detected.

Now we can incorporate this function into our main function:

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    dotenv().ok();
    let arguments = Args::parse();
    // ...
    let client = Client::new();
    let operating_system = get_pretty_name().unwrap_or("Linux".to_owned());

    // ...
}

We call get_pretty_name() and store its result in the operating_system variable.

Step 4: Including the operating system in our query

Now that we have detected the user's operating system, we can include it in our OpenAI API query:

let system_message = format!(
    "Act as a terminal expert, answer should be the COMMAND ONLY, no need to explain. OS: {OS}",
    OS = operating_system
);

let body = json!(
    {
        "model":"gpt-3.5-turbo",
        "messages":[
            {"role": "system",
            "content": system_message
            },
        {
            "role":"user",
            "content": query,
        }],
        "max_tokens": tokens,
    }
);

We create a system_message variable that includes the operating_system name, and we update the JSON payload to use this new message.

The format! macro in Rust is used to create a formatted string. It works similarly to println!, but instead of printing the formatted output to the console, it returns the result as a String. The first argument of format! is a string containing placeholders, denoted by curly braces {}, that are replaced with corresponding values provided as additional arguments. You can also include formatting options within the braces for more control over the output. The arguments are separated by commas and are evaluated in the order they appear in the format string. The resulting String can be used anywhere a regular string is needed, making format! a versatile tool for creating custom strings based on various inputs and formatting requirements.

And that's it! Our command line tool now accepts an optional max_tokens argument and detects the user's operating system to improve the quality of generated responses. Keep following this blog series as we continue our journey through Rust and build even more features into our command line tool! 🚀

Repository: Termoil