Ctrl+K

Prompts and guidance

Prompts and guidance#

Langworks employs templatable prompts and guidance when interfacing with LLMs. These templates are implemented using the Jinja template language, extended specifically for the challenges presented by prompting. Langworks differentiates between ‘static’ and ‘dynamic’ templates, respectively used by prompts and guidance.

Static templates#

Prompts rely on static templates. These templates largely work the same as Jinja templates. A notable difference is that Langworks’ templates are nestable. Consider the following example:

q = Query(

    query = (
        "The following {{language}} document, delimited by triple backticks, contains"
        " information about {{topic}}. Your task is to {{task}}. Base your work on the document"
        " provided."
        "\n\n"
        "```\n"
        "{{doc}}\n"
        "```"
    )

)

for task in [
    "explain what {{topic}} are",
    "explain how {{topic}} relate to humans",
    "give a taxonomy of {{topic}}"
]:

    q.exec(context = dict(
        language = "English",
        topic    = "dogs",
        task     = task,
        doc      = "..." # Contents of Wikipedia page about dogs.
    ))

Regular Jinja templates would fail to expand the tags embedded in the task variable. This would require the use of custom filters. Instead, Langworks renders recursively, allowing to use templates as variables. This allows these templates to be nested in Langworks, making it possible to take a modular approach to prompt design.

Dynamic templates#

Guidance is specified using dynamic templates. These templates enjoy the same features as static templates, being based on Jinja templates, and allowing for nesting of templates. However, dynamic templates have some additional features to put constraints on LLM generation.

Constraints#

By default Langworks applies no constraints on guidance. LLMs are simply requested to generate a response to the query prompted. However, when guidance is specified, constraints need to be specified too. Consider the following example:

query = Query(

    query = (
        "Your task is to solve the following equation: 1 + 1 ="
    ),

    guidance = (
        "1 + 1 ="
    )
)

Langworks cannot deduce the desired output. Perhaps a digit would suffice, but perhaps a further explanation is also desired. To help guide the generation of the desired output, constraints are required:

guidance = (
    "1 + 1 = {% regex = '\d' %}"
)

In this example, only a single digit is accepted, after which generation is stopped. If this was not desidered, guidance may have been specified as follows:

guidance = (
    "1 + 1 = {% regex = '\d' %}. This can be explained as follows: {% gen %}"
)

Accordingly, constraints serve as a powerful tool in guiding LLM generation. Langworks provides a number of constraints by default.

Base constraint#

All constraints derive from the base constraint. This constraint specifies the general format of all other constraints. Generally, constraints may be recognized by the {% and %} tags:

{% gen %}
{% choice %}
{% regex %}
{% json %}
{% grammar %}

Like regular Jinja tags, these constraints may take additional variables, namely:

var: str#

Name of the variable wherein the generated content is stored. Any subsequent template tags in the template containing the constraint, may access this variable using the output construct:

Query(

    query = (
        "Who or what is more popular: dogs or cats?"
    ),

    guidance = (
        "{% choice ['Dogs', 'Cats'], var = 'selection' %}, namely because"
        " {{ output.selection | lower }} {% gen %}"
    )
)

When the Query encompassing the constraint is embedded in a Langwork, connected Query objects will include the variable in their rendering context. Connected Nodes may access the variable through the context value returned by Query (also see: langworks.Query.exec()).

params: middleware.generic.SamplingParams#

Sampling parameters to apply when generating the content specified by this constraint. These parameters may be passed through the context, or by invoking the Params constructor within the template itself:

{% gen params = Params(temperature = 0.5) %}

If middleware requires any middleware specific parameters, the middleware ensures that these parameters are made accessible through the Params constructor.

Generation (gen)#

The gen constraint is actually a non-constraint. It simply requests the LLM to expand upon the current guidance. They may be embedded in guidance as follows:

Today I'm feeling {% gen %}

Choice (choice)#

The choice constraint requests the LLM to select an option from a limited list of options.

options: Sequence[str]#

The list of options that the LLM may choose from.

Regex (regex)#

The regex constraint requests the LLM to generate content satisfying the given regular expression.

spec: str#

The regular expression that specifies the constraint.

Tip

The Jinja template language does not parse raw string literals. Therefore, you are advised to pass any complex regular expressions through the context:

Query(

    query = (
        "Today it is January 1st, 2025. How may I write this date in YYYY-MM-DD format?"
    ),

    guidance = (
        "Today is: {% regex DATE %}"
    ),

    context = dict(
        DATE = r"\d\d\d\d\-\d\d\-\d\d"
    )
)

JSON (json)#

The json constraint requests the LLM to generate a JSON object satisfying the JSON schema passed. Given the potential complexity of such schema, json-tags may be defined as blocks:

{% json %}
{
    "type": "object",
    "properties": {
        "name": {
            "type": "string"
        },
        "description": {
            "type": "string"
        }
    }
}
{% endjson %}
spec: str#

The JSON schema that specifies the constraint.

Grammar (grammar)#

The grammar constraint requests the LLM to generate content satisfying the given EBNF grammar:

Present the number '7' with two leading zeroes:
{% grammar %}
?start: "0" "0" NUMBER
%import common.NUMBER
{% endgrammar %}.
spec: str#

The EBNF grammar that the generated content must conform to.

Contents