Tag: tech

Automating scientific code documentation: a GPT-powered POC for streamlined workflows.

Introduction

Working on scientific papers often involves translating algorithms into scientific formulas, typically formatted in LaTeX. This process can be tedious and time-consuming, especially in large projects, as it requires constant back-and-forth between the code repository and the LaTeX document.

While working on a large repository of algorithms, I began exploring ways to streamline this workflow. My motivation arose from the inefficiency of manually converting complex algorithms into LaTeX-compatible formulas. A particular challenge was ensuring consistency across multiple documents, especially in projects where formulas required frequent updates. This led me to explore how automation could streamline repetitive tasks while improving accuracy.

For the remainder of this document, I will use both the term “algorithm” and “scientific code.” All images in this article, except for the cover image, were created by the author.

Goal

My goal was to transition from scientific code to a comprehensive document that introduces the purpose of the code, defines variables, presents scientific formulas, includes a generated example plot, and demonstrates the calculations for a specific example. The document would follow a predefined framework, combining static and dynamic elements to ensure both consistency and adaptability.

The framework I designed included the following structure:

1. Front Page
   A visually appealing cover with key details such as the title and author.
2. Table of Contents
   Automatically generated to provide an overview of the document’s content.
3. Brief Description of the Document
   An introduction outlining the purpose and scope of the document.
4. Algorithms
   A section dedicated to documenting each algorithm in detail. For each algorithm, the following subsections would be included:
   – Introduction: A brief overview of the algorithm’s purpose and context.
   – Variables: A clear definition of all variables used in the algorithm.
   – Formulas: A presentation of the key formulas derived from the algorithm.
   – Example: A worked example to illustrate the algorithm’s application, complete with a generated plot.
   – Code: The corresponding code snippet to support reproducibility.

This structure was designed to dynamically adapt based on the number of algorithms being documented, ensuring a consistent and professional presentation regardless of the document’s size or complexity.

Structuring the Repository

To achieve this goal, a well-organized repository was essential for enabling a scalable and efficient solution. The algorithm calculations were grouped into a dedicated folder, with files named using a consistent snake_case convention that matched the algorithm names.

To ensure clarity and support reuse, initial values for examples and the generated plots were stored in separate folders. These folders followed the same naming convention as the algorithms but with distinct suffixes to differentiate their purpose. This structure ensured that all components were easy to find and consistent with the overall framework of the project.

Leveraging GPT for Automation

At the core of this project is the use of GPT models to automate the conversion of algorithms into LaTeX. GPT’s strength lies in its ability to interpret the structure of generic, variable-rich code and transform it into human-readable explanations and precisely formatted scientific formulas. This automation significantly reduces the manual effort required, ensuring both accuracy and consistency across documents.

For this project, I will leverage OpenAI’s ChatGPT-4o model, renowned for its advanced ability to comprehend and generate structured content. To interact with OpenAI’s API, you must have an OPENAI_KEY set in your environment. Below is a simple Python function I use to fetch responses from the GPT model:

import os
from openai import OpenAI
from dotenv import load_dotenv

def ask_chat_gpt(prompt):
    load_dotenv()
    api_key = os.getenv("OPENAI_KEY") or exit("API key missing")
    client = OpenAI(api_key=api_key)
    response = client.chat.completions.create(
        model="gpt-4o", 
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Workflow

Overview of What the Code Does
This code automates the generation of structured LaTeX documentation for Python algorithms, complete with examples, plots, and Python code listings. Here’s an overview:

Prompt Creation for GPT

This section describes custom functions designed to generate detailed prompts for GPT, enabling the automated creation of LaTeX documentation:

• make_algo_doc_gpt_prompt: A function that creates prompts instructing GPT to generate LaTeX sections, including introductions, variable descriptions, formulas, and example subsections.
• make_algo_example_gpt_prompt: A function that generates prompts for creating LaTeX example sections, incorporating plots and example calculations.

Document Generation

These functions are responsible for processing the GPT-generated content and saving it as LaTeX files:

• make_algo_doc: A function that uses GPT outputs to generate LaTeX documentation for each algorithm and saves it as a .tex file.
• make_algo_example: A function that creates .tex files for example sections, including plots and example calculations.

LaTeX Assembly

• Uses the pylatex library to programmatically create a full LaTeX document.
• Adds a title page, metadata, and a table of contents.
• Includes an introduction section with an overview of the algorithms and their purpose.
• Creates a chapter for each algorithm with sections from make_algo_doc and make_algo_example, example plots, and Python code listings.

# Create and structure the LaTeX document programmatically
doc = Document(documentclass="report")

# Include preamble and metadata
doc.preamble.append(NoEscape(r'input{algo_docs/init.tex}'))  # Custom preamble
doc.append(NoEscape(r'input{algo_docs/title_page.tex}'))     # Title page
doc.append(NoEscape(r'tableofcontents'))  # Table of contents

# Add Introduction Chapter
with doc.create(Chapter('Introduction')):
    doc.append(
        'This document provides an overview of various algorithms, exploring their design, analysis, and application in computational problem-solving. '
        'The aim is to facilitate understanding of their mechanisms and significance across different domains.'
    )

# Add Algorithms Chapter
with doc.create(Chapter('Algorithms')):
    doc.append(
        'This chapter presents detailed analyses of various algorithms, highlighting their theoretical foundations, use cases, and practical insights. '
        'Each algorithm is accompanied by examples and visualizations to illustrate its functionality and potential limitations.'
    )

# Process each Python file in the 'python_code' directory
python_code_dir = "python_code/"
output_folder = "algo_docs/"
plot_folder = "plots/"

for filename in os.listdir(python_code_dir):
    if filename.endswith(".py"):  # Process only Python files
        algorithm_name = filename.replace(".py", "")
        formatted_name = algorithm_name.replace("_", " ").title()

        # Define paths for documentation files and plots
        document_path = os.path.join(output_folder, f"{algorithm_name}_doc.tex")
        example_path = os.path.join(output_folder, f"{algorithm_name}_example.tex")
        plot_path = os.path.join(plot_folder, f"{algorithm_name}_plot.png")
        python_code_path = os.path.join(python_code_dir, filename)

        print(f"Processing: {filename}")

        # Start a new page for each algorithm
        doc.append(NoEscape(r'newpage'))

        # Generate documentation and example files with GPT
        make_algo_doc(algorithm_name)
        make_algo_example(algorithm_name)

        # Insert generated LaTeX sections
        doc.append(NoEscape(rf'input{{{document_path}}}'))
        doc.append(NoEscape(rf'input{{{example_path}}}'))

        # Insert plot directly after example subsection
        if os.path.exists(plot_path):
            with doc.create(Figure(position='H')) as figure:
                figure.add_image(plot_path, width=NoEscape(r'textwidth'))
                figure.add_caption(f'Example plot for {formatted_name}.')

        # Add a subsection for the Python code listing
        with doc.create(Subsection('Code Listing')):
            doc.append(NoEscape(rf'lstinputlisting[language=Python]{{{python_code_path}}}'))

        # Add a page break for clarity
        doc.append(NoEscape(r'clearpage'))

# Generate the LaTeX file
tex_file = "programmatic_report"
doc.generate_tex(tex_file)

# Compile the LaTeX file to a PDF
subprocess.run(["pdflatex", f"{tex_file}.tex"])

PDF Compilation

• The assembled document is saved and compiled into a polished PDF using pdflatex.

Crafting Effective Prompts: The Core Challenge

One of the most challenging aspects of this project was designing and refining the prompts used to interact with GPT. The success of the entire process depended on the quality of the GPT-generated output, making the creation of effective prompts a critical task that required extensive time and experimentation.

The prompts needed to strike a delicate balance:

• Clarity: Precisely guiding GPT to produce structured LaTeX content, including sections, subsections, and mathematical equations, while leaving no ambiguity about the desired format.
• Adaptability: Ensuring the prompts could handle a wide variety of algorithms, ranging from simple calculations to complex implementations.
• Consistency: Achieving reliable, well-formatted, and accurate output, even for edge cases or unconventional code structures.

To address these challenges, I implemented dynamic prompting. This approach involved programmatically generating prompts tailored to the contents of each file. By providing GPT with relevant context and specific instructions, dynamic prompting ensured the output was both accurate and contextually appropriate for the given algorithm.

Through numerous iterations, the prompts evolved to become precise and flexible, forming the foundation of the automation process.
Example of a prompt for generating LaTeX code from a algorithm:

Generate LaTeX code from the provided Python code. Follow these guidelines:

1. **Document Structure**:
- Start with `\section{}` for the algorithm title.
- Add a `\subsection{Introduction}` for a brief overview of the algorithm.
- Include a `\subsection{Variables}` section that lists all variables with descriptions, using subscript notation (e.g., `v_{\text{earth}}`).
- Add a `\subsection{Formulas}` section presenting the code's logic as LaTeX formulas. Use subscripted symbols for variable names instead of copying Python variable names directly.

2. **Formatting Rules**:
- Ensure that the output includes **only** the LaTeX content, without `\documentclass`, `\usepackage`, `\begin{document}`, `\end{document}`, or any unrelated text.
- Do **not** include the triple backticks (e.g., ```latex or ```).
- Properly close all LaTeX environments (e.g., `\begin{align*}...\end{align*}`).
- Ensure all brackets, parentheses, and braces are matched correctly.
- Maintain consistent subscript notation for all variables.

3. **Important Notes**:
- **Do not** include any text or explanations outside the LaTeX code.
- Only the relevant LaTeX content for the `\section`, `\subsection`, `\begin{align*}`, and `\end{align*}` parts should be generated.
- Ensure no extra or unrelated LaTeX sections are added.

Example: Hohmann Transfer Orbit Calculation

The following demonstrates how the Hohmann Transfer Orbit Calculation algorithm is documented using GPT-generated LaTeX code. This algorithm calculates the velocity changes (delta-v) required to transfer a spacecraft from Earth’s orbit to Mars’s orbit. Below is the Python implementation of the algorithm:

def calculate_hohmann_transfer(earth_orbit_radius, mars_orbit_radius):
    # Gravitational constant for the Sun
    mu_sun = 1.32712440018e20

    # Orbital velocities of Earth and Mars
    v_earth = np.sqrt(mu_sun / earth_orbit_radius)
    v_mars = np.sqrt(mu_sun / mars_orbit_radius)

    # Semi-major axis of the transfer orbit
    transfer_orbit_semi_major_axis = (earth_orbit_radius + mars_orbit_radius) / 2

    # Transfer orbit velocities at Earth and Mars
    v_transfer_at_earth = np.sqrt(2 * mu_sun / earth_orbit_radius - mu_sun / transfer_orbit_semi_major_axis)
    v_transfer_at_mars = np.sqrt(2 * mu_sun / mars_orbit_radius - mu_sun / transfer_orbit_semi_major_axis)

    # Delta-v at Earth and Mars
    delta_v_earth = v_transfer_at_earth - v_earth
    delta_v_mars = v_mars - v_transfer_at_mars

    # Total delta-v for the transfer
    total_delta_v = abs(delta_v_earth) + abs(delta_v_mars)

    return delta_v_earth, delta_v_mars, total_delta_v

Using the GPT prompt with this code, I generated LaTeX subsections for the documentation. Below are the components created:

Introduction to the Algorithm
GPT generated a LaTeX explanation of the algorithm’s purpose, detailing how it calculates velocity changes for an efficient interplanetary transfer.

Variable Definitions
GPT provided a clear explanation of all variables used in the algorithm.

Formulas
The key formulas used in the algorithm were formatted into LaTeX by GPT.

Example Section
Using example values, GPT generated LaTeX code for a worked example.

Plot Generation
A plot of the transfer orbit was generated using the example values and included in the LaTeX document.

Code Listing
The algorithm’s source code was appended to the document for completeness at the end.

Results and Challenges

Initial experiments with this system have been promising. Using Python and GPT-4, I successfully automated the conversion of several algorithms into LaTeX documents. The results of this proof of concept (POC) can be explored in my GitHub repository, where all aspects of the project are available for review.

The repository includes the complete Python codebase, showcasing the custom functions used to generate LaTeX documentation and create GPT prompts. It also contains the detailed prompts themselves, illustrating how the system guides GPT in producing structured and accurate LaTeX content. Additionally, the repository features the final outputs, including both the LaTeX source files and the compiled PDF documents.

While the initial results have been promising, the process has not been without its challenges and valuable insights along the way::

• Formatting Challenges: Occasionally, GPT would produce incorrect LaTeX formatting, leading to errors during the PDF conversion process. Although this issue was rare, I experimented with a solution: resubmitting the LaTeX code to GPT and asking it to fix the formatting. While this approach was consistently successful, it was not implemented as part of the workflow.
• Code Comments: Adding clear comments within the code helped GPT understand the context better and generate more accurate LaTeX outputs.
• Inconsistent Results: GPT occasionally produced varying outputs for the same code and prompt, emphasizing its inherent variability and the importance of careful testing.
• Crafting Effective Prompts: Writing effective prompts was challenging. Overloading the prompt with too much detail, like examples, often caused GPT to miss smaller elements such as formatting or structure. I discovered that breaking down instructions step by step and using very small, focused examples helped GPT perform better. Keeping prompts concise and structured with bullet points ensured that each key instruction was clearly understood and executed.
• Domain-Specific Terminology: Fine-tuning GPT for specialized terms is an area requiring further improvement to enhance accuracy.
• Variable Definitions: Keeping LaTeX variable definations in algorithm and examples consistent was challenging. Adding GPT-generated variable definitions to later prompts helped maintain uniformity.

Despite its imperfections, the workflow has drastically reduced the time spent on documentation by automating much of the process. While minor reviews and adjustments are still needed, they represent only a fraction of the effort previously required. This proof of concept demonstrates the potential to generate polished documents without writing LaTeX manually, though further refinement is needed to enhance consistency, scalability, and adaptability. The results so far highlight the significant promise of this approach.

Improvements

• Develop Validation Mechanisms
  Implement cross-referencing of generated formulas against known standards or benchmarks to ensure accuracy and consistency.
• Expand Use Cases
  Test the workflow on larger, more diverse datasets to improve scalability and adaptability for various scientific domains.
• Enhance Visual Documentation
  Incorporate additional visual elements, such as flowcharts, by using GPT to generate XML documents or similar formats.
• Generate Plots and Examples with GPT
  Extend GPT’s functionality to create example plots directly, reducing the reliance on external plotting tools.
• Experiment with Different GPT Models
  Thus far, I have primarily used ChatGPT-4 due to its accessibility, but further research is needed to identify the optimal model for this task. Exploring models tailored for technical content or incorporating a Retrieval-Augmented Generation (RAG) approach with a database of diverse scientific papers could improve accuracy and relevance.
• Transition from Proof of Concept (POC) to Minimum Viable Product (MVP)
  Evolve the project from a proof of concept to a minimum viable product by adding robust error handling, scalability features, and user-focused refinements.

Conclusion

This project has proven the potential of GPT models to automate the creation of structured LaTeX documentation, significantly reducing the manual effort involved. It successfully generated professional-quality outputs, including formulas, plots, and structured examples. However, challenges such as inconsistent results, formatting issues, and variability in GPT’s output highlighted the need for refinement. Strategies like dynamic prompting, better code commenting, and iterative validation have helped address these issues, but some manual oversight remains necessary.

Despite these challenges, the workflow has shown clear benefits, streamlining the documentation process and saving considerable time. While the solution is not yet perfect, it represents a significant step toward automating complex documentation tasks, paving the way for future improvements in accuracy.

LinkedIn Profile — Peder Ward

GitHub — Peder Ward

---

From Code to Paper: Using GPT Models and Python to Generate Scientific LaTeX Documents was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Automating scientific code documentation: a GPT-powered POC for streamlined workflows.

Introduction

Working on scientific papers often involves translating algorithms into scientific formulas, typically formatted in LaTeX. This process can be tedious and time-consuming, especially in large projects, as it requires constant back-and-forth between the code repository and the LaTeX document.

While working on a large repository of algorithms, I began exploring ways to streamline this workflow. My motivation arose from the inefficiency of manually converting complex algorithms into LaTeX-compatible formulas. A particular challenge was ensuring consistency across multiple documents, especially in projects where formulas required frequent updates. This led me to explore how automation could streamline repetitive tasks while improving accuracy.

For the remainder of this document, I will use both the term “algorithm” and “scientific code.” All images in this article, except for the cover image, were created by the author.

Goal

My goal was to transition from scientific code to a comprehensive document that introduces the purpose of the code, defines variables, presents scientific formulas, includes a generated example plot, and demonstrates the calculations for a specific example. The document would follow a predefined framework, combining static and dynamic elements to ensure both consistency and adaptability.

The framework I designed included the following structure:

1. Front Page
   A visually appealing cover with key details such as the title and author.
2. Table of Contents
   Automatically generated to provide an overview of the document’s content.
3. Brief Description of the Document
   An introduction outlining the purpose and scope of the document.
4. Algorithms
   A section dedicated to documenting each algorithm in detail. For each algorithm, the following subsections would be included:
   – Introduction: A brief overview of the algorithm’s purpose and context.
   – Variables: A clear definition of all variables used in the algorithm.
   – Formulas: A presentation of the key formulas derived from the algorithm.
   – Example: A worked example to illustrate the algorithm’s application, complete with a generated plot.
   – Code: The corresponding code snippet to support reproducibility.

This structure was designed to dynamically adapt based on the number of algorithms being documented, ensuring a consistent and professional presentation regardless of the document’s size or complexity.

Structuring the Repository

To achieve this goal, a well-organized repository was essential for enabling a scalable and efficient solution. The algorithm calculations were grouped into a dedicated folder, with files named using a consistent snake_case convention that matched the algorithm names.

To ensure clarity and support reuse, initial values for examples and the generated plots were stored in separate folders. These folders followed the same naming convention as the algorithms but with distinct suffixes to differentiate their purpose. This structure ensured that all components were easy to find and consistent with the overall framework of the project.

Leveraging GPT for Automation

At the core of this project is the use of GPT models to automate the conversion of algorithms into LaTeX. GPT’s strength lies in its ability to interpret the structure of generic, variable-rich code and transform it into human-readable explanations and precisely formatted scientific formulas. This automation significantly reduces the manual effort required, ensuring both accuracy and consistency across documents.

For this project, I will leverage OpenAI’s ChatGPT-4o model, renowned for its advanced ability to comprehend and generate structured content. To interact with OpenAI’s API, you must have an OPENAI_KEY set in your environment. Below is a simple Python function I use to fetch responses from the GPT model:

import os
from openai import OpenAI
from dotenv import load_dotenv

def ask_chat_gpt(prompt):
    load_dotenv()
    api_key = os.getenv("OPENAI_KEY") or exit("API key missing")
    client = OpenAI(api_key=api_key)
    response = client.chat.completions.create(
        model="gpt-4o", 
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Workflow

Overview of What the Code Does
This code automates the generation of structured LaTeX documentation for Python algorithms, complete with examples, plots, and Python code listings. Here’s an overview:

Prompt Creation for GPT

This section describes custom functions designed to generate detailed prompts for GPT, enabling the automated creation of LaTeX documentation:

• make_algo_doc_gpt_prompt: A function that creates prompts instructing GPT to generate LaTeX sections, including introductions, variable descriptions, formulas, and example subsections.
• make_algo_example_gpt_prompt: A function that generates prompts for creating LaTeX example sections, incorporating plots and example calculations.

Document Generation

These functions are responsible for processing the GPT-generated content and saving it as LaTeX files:

• make_algo_doc: A function that uses GPT outputs to generate LaTeX documentation for each algorithm and saves it as a .tex file.
• make_algo_example: A function that creates .tex files for example sections, including plots and example calculations.

LaTeX Assembly

• Uses the pylatex library to programmatically create a full LaTeX document.
• Adds a title page, metadata, and a table of contents.
• Includes an introduction section with an overview of the algorithms and their purpose.
• Creates a chapter for each algorithm with sections from make_algo_doc and make_algo_example, example plots, and Python code listings.

# Create and structure the LaTeX document programmatically
doc = Document(documentclass="report")

# Include preamble and metadata
doc.preamble.append(NoEscape(r'input{algo_docs/init.tex}'))  # Custom preamble
doc.append(NoEscape(r'input{algo_docs/title_page.tex}'))     # Title page
doc.append(NoEscape(r'tableofcontents'))  # Table of contents

# Add Introduction Chapter
with doc.create(Chapter('Introduction')):
    doc.append(
        'This document provides an overview of various algorithms, exploring their design, analysis, and application in computational problem-solving. '
        'The aim is to facilitate understanding of their mechanisms and significance across different domains.'
    )

# Add Algorithms Chapter
with doc.create(Chapter('Algorithms')):
    doc.append(
        'This chapter presents detailed analyses of various algorithms, highlighting their theoretical foundations, use cases, and practical insights. '
        'Each algorithm is accompanied by examples and visualizations to illustrate its functionality and potential limitations.'
    )

# Process each Python file in the 'python_code' directory
python_code_dir = "python_code/"
output_folder = "algo_docs/"
plot_folder = "plots/"

for filename in os.listdir(python_code_dir):
    if filename.endswith(".py"):  # Process only Python files
        algorithm_name = filename.replace(".py", "")
        formatted_name = algorithm_name.replace("_", " ").title()

        # Define paths for documentation files and plots
        document_path = os.path.join(output_folder, f"{algorithm_name}_doc.tex")
        example_path = os.path.join(output_folder, f"{algorithm_name}_example.tex")
        plot_path = os.path.join(plot_folder, f"{algorithm_name}_plot.png")
        python_code_path = os.path.join(python_code_dir, filename)

        print(f"Processing: {filename}")

        # Start a new page for each algorithm
        doc.append(NoEscape(r'newpage'))

        # Generate documentation and example files with GPT
        make_algo_doc(algorithm_name)
        make_algo_example(algorithm_name)

        # Insert generated LaTeX sections
        doc.append(NoEscape(rf'input{{{document_path}}}'))
        doc.append(NoEscape(rf'input{{{example_path}}}'))

        # Insert plot directly after example subsection
        if os.path.exists(plot_path):
            with doc.create(Figure(position='H')) as figure:
                figure.add_image(plot_path, width=NoEscape(r'textwidth'))
                figure.add_caption(f'Example plot for {formatted_name}.')

        # Add a subsection for the Python code listing
        with doc.create(Subsection('Code Listing')):
            doc.append(NoEscape(rf'lstinputlisting[language=Python]{{{python_code_path}}}'))

        # Add a page break for clarity
        doc.append(NoEscape(r'clearpage'))

# Generate the LaTeX file
tex_file = "programmatic_report"
doc.generate_tex(tex_file)

# Compile the LaTeX file to a PDF
subprocess.run(["pdflatex", f"{tex_file}.tex"])

PDF Compilation

• The assembled document is saved and compiled into a polished PDF using pdflatex.

Crafting Effective Prompts: The Core Challenge

One of the most challenging aspects of this project was designing and refining the prompts used to interact with GPT. The success of the entire process depended on the quality of the GPT-generated output, making the creation of effective prompts a critical task that required extensive time and experimentation.

The prompts needed to strike a delicate balance:

• Clarity: Precisely guiding GPT to produce structured LaTeX content, including sections, subsections, and mathematical equations, while leaving no ambiguity about the desired format.
• Adaptability: Ensuring the prompts could handle a wide variety of algorithms, ranging from simple calculations to complex implementations.
• Consistency: Achieving reliable, well-formatted, and accurate output, even for edge cases or unconventional code structures.

To address these challenges, I implemented dynamic prompting. This approach involved programmatically generating prompts tailored to the contents of each file. By providing GPT with relevant context and specific instructions, dynamic prompting ensured the output was both accurate and contextually appropriate for the given algorithm.

Through numerous iterations, the prompts evolved to become precise and flexible, forming the foundation of the automation process.
Example of a prompt for generating LaTeX code from a algorithm:

Generate LaTeX code from the provided Python code. Follow these guidelines:

1. **Document Structure**:
- Start with `\section{}` for the algorithm title.
- Add a `\subsection{Introduction}` for a brief overview of the algorithm.
- Include a `\subsection{Variables}` section that lists all variables with descriptions, using subscript notation (e.g., `v_{\text{earth}}`).
- Add a `\subsection{Formulas}` section presenting the code's logic as LaTeX formulas. Use subscripted symbols for variable names instead of copying Python variable names directly.

2. **Formatting Rules**:
- Ensure that the output includes **only** the LaTeX content, without `\documentclass`, `\usepackage`, `\begin{document}`, `\end{document}`, or any unrelated text.
- Do **not** include the triple backticks (e.g., ```latex or ```).
- Properly close all LaTeX environments (e.g., `\begin{align*}...\end{align*}`).
- Ensure all brackets, parentheses, and braces are matched correctly.
- Maintain consistent subscript notation for all variables.

3. **Important Notes**:
- **Do not** include any text or explanations outside the LaTeX code.
- Only the relevant LaTeX content for the `\section`, `\subsection`, `\begin{align*}`, and `\end{align*}` parts should be generated.
- Ensure no extra or unrelated LaTeX sections are added.

Example: Hohmann Transfer Orbit Calculation

The following demonstrates how the Hohmann Transfer Orbit Calculation algorithm is documented using GPT-generated LaTeX code. This algorithm calculates the velocity changes (delta-v) required to transfer a spacecraft from Earth’s orbit to Mars’s orbit. Below is the Python implementation of the algorithm:

def calculate_hohmann_transfer(earth_orbit_radius, mars_orbit_radius):
    # Gravitational constant for the Sun
    mu_sun = 1.32712440018e20

    # Orbital velocities of Earth and Mars
    v_earth = np.sqrt(mu_sun / earth_orbit_radius)
    v_mars = np.sqrt(mu_sun / mars_orbit_radius)

    # Semi-major axis of the transfer orbit
    transfer_orbit_semi_major_axis = (earth_orbit_radius + mars_orbit_radius) / 2

    # Transfer orbit velocities at Earth and Mars
    v_transfer_at_earth = np.sqrt(2 * mu_sun / earth_orbit_radius - mu_sun / transfer_orbit_semi_major_axis)
    v_transfer_at_mars = np.sqrt(2 * mu_sun / mars_orbit_radius - mu_sun / transfer_orbit_semi_major_axis)

    # Delta-v at Earth and Mars
    delta_v_earth = v_transfer_at_earth - v_earth
    delta_v_mars = v_mars - v_transfer_at_mars

    # Total delta-v for the transfer
    total_delta_v = abs(delta_v_earth) + abs(delta_v_mars)

    return delta_v_earth, delta_v_mars, total_delta_v

Using the GPT prompt with this code, I generated LaTeX subsections for the documentation. Below are the components created:

Introduction to the Algorithm
GPT generated a LaTeX explanation of the algorithm’s purpose, detailing how it calculates velocity changes for an efficient interplanetary transfer.

Variable Definitions
GPT provided a clear explanation of all variables used in the algorithm.

Formulas
The key formulas used in the algorithm were formatted into LaTeX by GPT.

Example Section
Using example values, GPT generated LaTeX code for a worked example.

Plot Generation
A plot of the transfer orbit was generated using the example values and included in the LaTeX document.

Code Listing
The algorithm’s source code was appended to the document for completeness at the end.

Results and Challenges

Initial experiments with this system have been promising. Using Python and GPT-4, I successfully automated the conversion of several algorithms into LaTeX documents. The results of this proof of concept (POC) can be explored in my GitHub repository, where all aspects of the project are available for review.

The repository includes the complete Python codebase, showcasing the custom functions used to generate LaTeX documentation and create GPT prompts. It also contains the detailed prompts themselves, illustrating how the system guides GPT in producing structured and accurate LaTeX content. Additionally, the repository features the final outputs, including both the LaTeX source files and the compiled PDF documents.

While the initial results have been promising, the process has not been without its challenges and valuable insights along the way::

• Formatting Challenges: Occasionally, GPT would produce incorrect LaTeX formatting, leading to errors during the PDF conversion process. Although this issue was rare, I experimented with a solution: resubmitting the LaTeX code to GPT and asking it to fix the formatting. While this approach was consistently successful, it was not implemented as part of the workflow.
• Code Comments: Adding clear comments within the code helped GPT understand the context better and generate more accurate LaTeX outputs.
• Inconsistent Results: GPT occasionally produced varying outputs for the same code and prompt, emphasizing its inherent variability and the importance of careful testing.
• Crafting Effective Prompts: Writing effective prompts was challenging. Overloading the prompt with too much detail, like examples, often caused GPT to miss smaller elements such as formatting or structure. I discovered that breaking down instructions step by step and using very small, focused examples helped GPT perform better. Keeping prompts concise and structured with bullet points ensured that each key instruction was clearly understood and executed.
• Domain-Specific Terminology: Fine-tuning GPT for specialized terms is an area requiring further improvement to enhance accuracy.
• Variable Definitions: Keeping LaTeX variable definations in algorithm and examples consistent was challenging. Adding GPT-generated variable definitions to later prompts helped maintain uniformity.

Despite its imperfections, the workflow has drastically reduced the time spent on documentation by automating much of the process. While minor reviews and adjustments are still needed, they represent only a fraction of the effort previously required. This proof of concept demonstrates the potential to generate polished documents without writing LaTeX manually, though further refinement is needed to enhance consistency, scalability, and adaptability. The results so far highlight the significant promise of this approach.

Improvements

• Develop Validation Mechanisms
  Implement cross-referencing of generated formulas against known standards or benchmarks to ensure accuracy and consistency.
• Expand Use Cases
  Test the workflow on larger, more diverse datasets to improve scalability and adaptability for various scientific domains.
• Enhance Visual Documentation
  Incorporate additional visual elements, such as flowcharts, by using GPT to generate XML documents or similar formats.
• Generate Plots and Examples with GPT
  Extend GPT’s functionality to create example plots directly, reducing the reliance on external plotting tools.
• Experiment with Different GPT Models
  Thus far, I have primarily used ChatGPT-4 due to its accessibility, but further research is needed to identify the optimal model for this task. Exploring models tailored for technical content or incorporating a Retrieval-Augmented Generation (RAG) approach with a database of diverse scientific papers could improve accuracy and relevance.
• Transition from Proof of Concept (POC) to Minimum Viable Product (MVP)
  Evolve the project from a proof of concept to a minimum viable product by adding robust error handling, scalability features, and user-focused refinements.

Conclusion

This project has proven the potential of GPT models to automate the creation of structured LaTeX documentation, significantly reducing the manual effort involved. It successfully generated professional-quality outputs, including formulas, plots, and structured examples. However, challenges such as inconsistent results, formatting issues, and variability in GPT’s output highlighted the need for refinement. Strategies like dynamic prompting, better code commenting, and iterative validation have helped address these issues, but some manual oversight remains necessary.

Despite these challenges, the workflow has shown clear benefits, streamlining the documentation process and saving considerable time. While the solution is not yet perfect, it represents a significant step toward automating complex documentation tasks, paving the way for future improvements in accuracy.

LinkedIn Profile — Peder Ward

GitHub — Peder Ward

---

From Code to Paper: Using GPT Models and Python to Generate Scientific LaTeX Documents was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models

Traditional RAG vs. dynamic RAG

In this article, I explore the fundamental concepts explained in the research paper titled “DRAGIN : Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models” by Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. This paper can be accessed here.

Introduction — Lets look at a short story!

Imagine you’re working on a problem. At the very beginning, you get only one chance to ask your professor for guidance. This means it’s important to understand the entire scope of the problem upfront. If it’s a simple problem, that might be fine — you ask your question, get clarity, and move forward.

Now, imagine that the problem is much more complex. The more you dive into it, the more questions you have! Unfortunately, you can’t go back to your professor because all your questions had to be asked at the start. This makes solving the problem much harder.

But what if, instead, you were allowed to go back to your professor every time you discovered a new question that expanded the scope of the problem? This approach lets you navigate the complexity iteratively, asking for guidance whenever the problem evolves. That is the essence of DRAGIN (Dynamic RAG) over Traditional RAG.

And given how complex and multi-dimensional our tasks, problems, and worlds have become, the need for this dynamic approach is greater than ever!

Large Language models have changed the way we all access information. We are at that point where the way we search has forever changed. Now, instead of finding multiple links and processing the information to answer our questions, we can directly ask the LLM!

However, there are still a number of issues :

1. Hallucinations : Generating fabricated information
2. Datedness/Staleness : Inability to incorporate up to date information
3. Proprietary Information : Inaccessibility to specialised knowledge

To overcome the above issues, Retrieval Augmented Generation (RAG) emerged as a promising solution. The way it works, is by accessing and incorporating the relevant external information needed for the LLM to generate accurate responses.

However with traditional RAG methods, they rely on single-round retrieval, which would mean retrieving external information once, at the start of the information generation. This works well for straightforward tasks, however our needs and requirements from LLMs are getting more complex, multi-step and requiring longer responses.

In these cases, a single-round of retrieval will not work well and mutiple rounds of retrieval need to be conducted. Now when, we talk about retrieving information more than once, the two next questions are :

When to retrieve and What to retrieve? To solve these, a number of RAG methods have been devised :

Fixed Retrieval Methods :

IRCoT (Fixed Sentence RAG) [1] : Retrieval is conducted for each generated query and the latest sentence is used as a query.

RETRO [2] and IC-RALM [3] (Fixed Length RAG) : A sliding window is defined and the retrieval module is triggered every n tokens.

But aren’t we retrieving too often and hence retrieving information that may be unnecessary ? This would introduce noise and could jeopardize the quality of the LLM’s outputs, which defeats the original purpose of improving accuracy. These rules are still static and we need to think of dynamic ways of retrieval.

Dynamic Retrieval Methods :

FLARE [4] (Low Confidence RAG) : Retrieval is conducted dynamically, when the LLM’s confidence (the generation probability) on the next token is lower than certain thresholds. So FLARE, is triggering retrieval based on uncertainty.

For determining what to retrieve, the LLMs often restrict themselves to queries based on the last few generated tokens or sentences. These methods of query formulation for retrieval may not work, when the tasks get more complex and the LLM’s information needs span over the entire context!

Finally, moving on to the star of the show : DRAGIN!

DRAGIN (Dynamic Retrieval Augmented Generation based on Information Needs) :

This method is specifically designed to make decisions about when and what to retrieve to cater to the LLM’s information needs. So, it optimizes the process of information retrieval using two frameworks. As the authors explain in their paper, DRAGIN has two key frameworks :

I. RIND (Real-time Information Needs Detection) : When to retrieve ?

It considers the LLM’s uncertainty about its own content, the influence of each token and the semantics of each token.

II. QFS (Query Formulation based on Self-Attention) : What to retrieve?

Query formulation leverages the LLM’s self-attention across the entire context, rather than not just the last few tokens or sentences.

Illustration of the DRAGIN framework

To illustrate the above frameworks, the paper uses an example query about the ‘brief introduction to Einstein’.

Explanation :

Input is Provided: The system is queried to provide some introduction about Einstein.

Processing Starts: The system begins generating a response based on what it knows. It uses the RIND module to decide if it has enough information or if it needs to look things up.

Checking for Required Information (RIND): The system breaks down the query into smaller parts (tokens), like “position,” “at,” “University,” etc. It checks which parts (tokens) need more information. For example, “university” might need additional data because it’s not specific enough.

Triggering Retrieval: If a token like “university” is considered to be important and unclear, the system triggers retrieval to gather external information about it. In this case, it looks up relevant data about Einstein and universities.

Formulating the Query (QFS): The system uses its self attention mechanism to determine which words are most relevant for forming a precise query. For example, it might pick “Einstein,” “1903,” and “secured a job” as the key parts.

These keywords are used to craft a query, such as “Einstein 1903 secured a job,” which is sent to an external source for information.

Retrieving and Adding Information: The external source provides the required details. For example, it might return, “In 1903, Einstein secured a job at the Swiss Patent Office.” The system incorporates this new information into the response.

Continuing Generation: With the new details, the system continues generating a more complete and accurate response.For example, it might now say, “In 1903, Einstein secured a job at the Swiss Patent Office. This allowed him to have a stable income.”

Repeating the Process: If more requirements are identified, the process repeats: checking, retrieving, and integrating information until the response is complete and accurate. This process ensures that the system can dynamically fill in gaps in its knowledge and provide detailed, accurate answers by combining what it knows with retrieved external information.

Detailed Methodology about RAG

The frameworks mentioned in the paper are:

A. Real-time Information Needs Detection (RIND) : Retrieval is triggered based on uncertainty of tokens, influence on other tokens and semantic significance of each token.

i. The uncertainty of each token generated by the LLM, is quantified. This is done by calculating the entropy of the token’s probability distribution across the vocabulary. Consider an output sequence T = {t1,t2,…tn}, with each ti representing an individual token at the position i. For any token ti, the entropy is calculated as follows :

where pi(v) denotes the probability of generating the token v over all the tokens in the vocabulary.

ii. The influence of each token on subsequent tokens, is done by leveraging the self-attention scores. For a token t, the max attention value is identified

iii. The semantic contribution of each token ti, a binary indicator is employed. This filters out stop words.

Combining the uncertainty, significance and semantics, RIND computes a score and if this is greater than a pre-defined threshold then retrieval is triggered.

B. Query Formulation based on Self-Attention (QFS)

Once retrieval is triggered, the next step is to formulate an efficient query from external databases for the continued generation of LLMs. In the existing dynamic RAG frameworks, queries are formulated using the last sentence or last tokens generated by the LLM. This narrow scope doesn’t capture the need for real-time information needs. It examines the full context.

Suppose RIND identifies the token ti at position i, requires external knowledge and triggers retrieval.

Since the token ti was generated based on based on the knowledge of all the preceding tokens, it only makes sense to look at the entire content generated, until now, to formulate a query. It uses the following steps:

Step 1: Extracts the attention scores of the last transformer layer for each token ti.

Step 2: Sorts the attention scores in descending order, to identify the top n scores. (This is basically, identifying the most important tokens).

Step 3: Finds the words corresponding to these tokens from the vocabulary and arranges them in their original order. (This brings back the structure of the language form the attention scores and tokens).

Step 4 : Construct the query Qi using the words associated with these top n tokens.

C. Continue Generation after Retrieval

Once RIND has identified the position i, at which external knowledge is needed and QFS creates the query, Qi to extract the information using an off-the-shelf retrieval model (e.g. BM25).

It finds the relevant information in documents Di1, Di2 and Di3. It integrates the relevant information at position i, by truncating the LLM’s output. This retrieved knowledge is integrated using the following designed prompt template.

Limitations

As the paper states, the primary limitations of this paper is the reliance on the self-attention mechanisms for both the Real-time Information Needs Detection (RIND) and Query Formulation based on Self-Attention (QFS). While self-attention scores are available for all source LLMs, it is not applicable for certain APIs that do not provide access to self-attention scores.

A point worth considering is the impact on inference time latency and cost: in the paper, the authors point out that these are only marginally more since an imperfect token sub-sequence is rapidly detected, and further generation is interrupted until remediation.

Conclusion

The DRAGIN framework allows us to look to move a few steps ahead of the traditional RAG framework. It allows us to perform multiple retrievals, based on the information needs of generation. It is an optimized framework for multiple retrievals!

Our needs and requirements from LLMs are becoming larger and more complex, and in such cases where we want to retrieve information accurately, with just the right number of retrievals.

To conclude, DRAGIN :

Strikes the perfect balance for the number of retrievals.
Produces highly context-aware queries for retrievals.
Generates content from the LLMs with better accuracy!

Thank you so much for reading and for a more detailed explanation of the research paper, please watch my video!

References :

[1] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509.

[2] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022.

[3] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. arXiv preprint arXiv:2302.00083.

[4] Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983.

---

DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language… was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, we demonstrate how you can use Amazon SageMaker HyperPod training plans, to bring down your training cluster procurement wait time. We guide you through a step-by-step implementation on how you can use the (AWS CLI) or the AWS Management Console to find, review, and create optimal training plans for your specific compute and timeline needs. We further guide you through using the training plan to submit SageMaker training jobs or create SageMaker HyperPod clusters.