
# PDL Language Tutorial

The following sections give a step-by-step overview of PDL language features.
All the examples in this tutorial can be found in `examples/tutorial`.

## Simple text

The simplest PDL program is one that generates a small text ([file](../examples/tutorial/simple_program.pdl)):

```yaml
--8<-- "./examples/tutorial/simple_program.pdl"
```

This program has a `description` field, which contains a title. The `description` field is optional. It also has a `text` field, which can be either a string, a *block*, or a list of strings and blocks. A block is a recipe for how to obtain data (e.g., model call, code call, etc...). In this case, there are no calls to an LLM or other tools, and `text` consists of a simple string.

To render the program into an actual text, we have a PDL interpreter that can be invoked as follows:

```
pdl examples/tutorial/simple_program.pdl
```

This results in the following output:

```
Hello, world!
```

## Calling an LLM

```yaml
--8<-- "./examples/tutorial/calling_llm.pdl"
```

In this program ([file](../examples/tutorial/calling_llm.pdl)), the `text` starts with the word `"Hello\n"`, and we call a model (`ollama/granite3.2:2b`) with this as input prompt.
The model is passed a parameter `stop` to indicate the stop sequences.

A PDL program computes 2 data structures. The first is a JSON corresponding to the result of the overall program, obtained by aggregating the results of each block. This is what is printed by default when we run the interpreter. The second is a conversational background context, which is a list of role/content pairs (list of messages), where we implicitly keep track of roles and content for the purpose of communicating with models that support chat APIs. The contents in the latter correspond to the results of each block. The conversational background context is what is used to make calls to LLMs via LiteLLM.

In this example, since the `input` field is not specified in the model call, the entire text up to that point is passed to the model as input context, using the
default role `user`.

When we execute this program using the interpreter, we obtain:

```
Hello
Hello
```

where the second `Hello` has been generated by Granite.

Here's another example of model call that includes an `input` field ([file](../examples/tutorial/calling_llm_with_input.pdl)):

```yaml
--8<-- "./examples/tutorial/calling_llm_with_input.pdl"
```

In this case, the input passed to the model is the sentence: `Translate the word 'Hello' to French` and nothing else from the surrounding document. When we execute this program, we obtain:

```
Hello
Bonjour (pronounced bon-zhoor) is the translation for "Hello" in French. It's an informal greeting used during the day, similar to how we use "Hi" or "Hello." For a more formal context, you might say "Bonjour," which means "Good day."
```

where the second line is generated by the model.

Using the `input` field, we can also give a directly an array of messages (`role`/`content`) to the model ([file](../examples/tutorial/calling_llm_with_input_messages.pdl)):

```yaml
--8<-- "./examples/tutorial/calling_llm_with_input_messages.pdl"
```

This has the same output as the previous program. An alternative way of writing this is [this](../examples/tutorial/calling_llm_with_input_messages_var.pdl) program.

### Parameter defaults for watsonx Granite models

When using Granite models, we use the following defaults for model parameters:

- `temperature`: 0
- `max_new_tokens`: 1024
- `min_new_tokens`: 1
- `repetition_penalty`: 1.05

  Also if the `decoding_method` is `sample` (`watsonx_text text completion endpoint), then the following defaults are used:

- `temperature`: 0.7
- `top_p`: 0.85
- `top_k`: 50

The user can override these defaults by explicitly including them in the model call.

## Variable Definition and Use

Any block can define a variable using a `def: <var>` field. This means that the output of that block is assigned to the variable `<var>`, which may be reused at a later point in the document.

Consider the following example ([file](../examples/tutorial/variable_def_use.pdl)):

```yaml
--8<-- "./examples/tutorial/variable_def_use.pdl"
```

Here we assign the output of the model to variable `GEN` using the `def` field. The last line of the program prints out the value of `GEN`. Notice the notation `${ }` for accessing the value of a variable. Any [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) expression is allowed to be used inside these braces. These expressions
are also used to specify conditions for loops and conditionals. See for example this [file](../examples/tutorial/programs/chatbot.pdl).

When we execute this program, we obtain:

```
Hello
Hello
GEN is equal to: Hello
```

## Model Chaining

In PDL, we can declaratively chain models together as in the following example ([file](../examples/tutorial/calling_llm_chaining.pdl)):

```yaml
--8<-- "./examples/tutorial/calling_llm_chaining.pdl"
```

In this program, the first call is to a Granite model with the prompt `"Hello\n"`. The following block in the program prints out the sentence: `"\nDid you just say Hello?\n"`. The final line of the program takes the entire context produced so far and passes it as input to the Granite model. Notice that the input passed to this model is the context up to that point, represented as a conversation. This makes it easy to chain models together and continue building on previous interactions. Notice how the conversational context is accumulated implicitly without requiring the user to explicitly manage messages.

When we execute this program, we obtain:

```
Hello
Hello
Did you just say Hello?
Yes, I did. That's how I greet people in this conversation. It's a common way to start a dialogue. How can I assist you today?
```

## Function Definition

PDL also supports function definitions to make it easier to reuse code.
Suppose we want to define a translation function that takes a string and calls a Granite model for the translation. This would be written in PDL as follows ([file](../examples/tutorial/function_definition.pdl)):

```yaml
--8<-- "./examples/tutorial/function_definition.pdl"
```

In this program, the first block defines a function `translate` that takes as parameters `sentence` and `language`, both of which are of type string. The body of the function is defined by its `return` field. In this case, we formulate a translation prompt using the parameters and send it to a Granite model.

The last two blocks are calls to this function, as indicated by `call: ${ translate }`. This block specifies the arguments to be passed. When we execute this program, we obtain:

```
'J'aime Paris !'
'Me encanta Madrid.'
```

A function only contributes to the result when it is called. So the definition itself results in `""`. When we call a function, we implicitly pass the current background context, and this is used as input to model calls inside the function body. In the above example, since the `input` field is omitted, the entire document produced at that point is passed as input to the Granite model.

To reset the context when calling a function, we can pass the special argument: `pdl_context: []`.

Notice that the arguments of function calls are expressions and cannot be arbitrary PDL blocks.

A function name can be aliased (see [example](../examples/tutorial/function_alias.pdl)).

The context inherited by a function can be reset at the call site (see [example](../examples/tutorial/function_empty_context.pdl)).

Functions can be declared with optional parameters (see [example](../examples/tutorial/function_optional_params.pdl)).

## Grouping Variable Definitions in Defs

In PDL, the above program can be written more neatly by grouping certain variable definitions into a `defs` section, as follows ([file](../examples/tutorial/defs.pdl)):

```yaml
--8<-- "./examples/tutorial/defs.pdl"
```

This program has the same output has the one from the previous section.

Any block can have a `defs` field defining variables used in that block. Notice it's different than the `def` field which stores the
result of the block after execution.

For another example, see [file](../examples/tutorial/defs-hello.pdl).

## Muting Block Output with contribute

By default, when a PDL block is executed it produces a result that is contributed to the overall result, and it also contributes to the background context. It is possible to mute both contributions by setting `contribute` to `[]` for any block. This feature allows the computation of intermediate values that are not necessarily output as a result. The value of the variable specified in `def` is still set to the result of the block.

Consider the similar example as above, but with `contribute` set to `[]` ([file](../examples/tutorial/muting_block_output.pdl)):

```yaml
--8<-- "./examples/tutorial/muting_block_output.pdl"
```

The call to the translator with French as language does not produce an output. However, we save the result in variable `FRENCH` and use it in the last sentence of the document. When we execute this program, we obtain:

```
The french sentence was: 'J'aime Paris !'
```

In general, `contribute` can be used to set how the result of the block contribute to the final result and the background context.
Here are its possible values:

- `[]`: no contribution to either the final result or the background context

- `[result]`: contribute to the final result but not the background context

- `[context]`: contribute to the background context but not the final result

- `[result, context]`: contribute to both, which is also the default setting.

## Specifying Data

In PDL, the user specifies step by step the shape of data they wish to generate. A `text` block takes a list of blocks, stringifies the result of each block,
and concatenates them.

An `array` takes a list of blocks and creates an array of the results of each block:

```yaml
array:
  - apple
  - orange
  - banana
```

This results in the following output:

```
["apple", "orange", "banana"]
```

Each list item can contain any PDL block (strings are shown here), and the overall result is presented as an array.

An `object` constructs an object:

```
object:
  name: Bob
  job: manager
```

This results in the following output:

```
{"name": "Bob", "job": "manager"}
```

Each value in the object can be any PDL block, and the result is presented as an object.

A `lastOf` is a sequence, where each block in the sequence is executed and the overall result is that of the last block.

```
lastOf:
  - 1
  - 2
  - 3
```

This results in the following output:

```
3
```

Each list item can contain any PDL block (strings are shown here), and the result of the whole list is that of the last block.

Notice that block types that require lists (`repeat`, `for`, `if-then-else`) need to specify the shape of data in their bodies, for
example `text` or `array`. It is a syntax error to omit it. For more detailed discussion on this see [this section](#conditionals-and-loops).

## Input from File or Stdin

PDL can accept textual input from a file or stdin. In the following example ([file](../examples/tutorial/input_file.pdl)), the contents of this [file](../examples/tutorial/data.txt) are read by PDL and incorporated in the document. The result is also assigned to a variable `HELLO`.

```yaml
--8<-- "./examples/tutorial/input_file.pdl"
```

In the next example, prompts are obtained from stdin ([file](../examples/tutorial/input_stdin.pdl)). This is indicated by assigning the value `null` to the `read` field.

```yaml
--8<-- "./examples/tutorial/input_stdin.pdl"
```

If the `message` field is omitted then one is provided for you.

The following example shows a multiline stdin input ([file](../examples/tutorial/input_stdin_multiline.pdl)). When executing this code and to exit from the multiline input simply press control D (on macOS).

```yaml
--8<-- "./examples/tutorial/input_stdin_multiline.pdl"
```

Finally, the following example shows reading content in JSON format.

Consider the JSON content in this [file](../examples/tutorial/input.json):

```yaml
--8<-- "./examples/tutorial/input.json"
```

The following PDL program reads this content and assigns it to variable `PERSON` in JSON format using the `parser` ([file](../examples/tutorial/input_file_json.pdl)). The reference `PERSON.address.street` then refers to that field inside the JSON object. Note that the PDL interpreter performs automatic repair of JSON objects generated by LLMs.

```yaml
--8<-- "./examples/tutorial/input_file_json.pdl"
```

When we execute this program, we obtain:

```
Bob lives at the following address:
87 Smith Road in the town of Armonk, NY
```

## Parsing the output of a block

As we saw in the previous section, it is possible to use the `parser: json` setting to parse the result of a block as a JSON.
Other possible values for `parser` are `yaml`, `jsonl`, or `regex`.

The following example extracts using a regular expression parser the code between triple backtick generated by a model:

```yaml
--8<-- "./examples/tutorial/parser_regex_code.pdl"
```

Here is another example using a regular expression:

```yaml
--8<-- "./examples/tutorial/parser_regex.pdl"
```

We support the following operations with the`regex` parser (indicated with the `mode` field):

- `fullmatch` (default)

- `search`

- `match`

- `split`

- `findall`

Here is an example using the `findall` mode that returns the list `['1', '2', '3', '4']`:

```yaml
--8<-- "./examples/tutorial/parser_findall.pdl"
```

See [here](https://docs.python.org/3/library/re.html) for more information on how to write regular expressions.

## Calling code

The following script shows how to execute python code ([file](../examples/tutorial/code_python.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`).  In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result.

In order to define variables that are carried over to the next Python code block, a special variable `PDL_SESSION` can be used, and
variables assigned to it as fields.
See for example: ([file](../examples/tutorial/programs/tfidf_rag.pdl)).

```yaml
--8<-- "./examples/tutorial/code_python.pdl"
```

This results in the following output (for example):

```
Hello, r!
```

PDL also supports Jinja code blocks, shell commands, as well as PDL code blocks for meta-cycle programming. For more examples, see
([Jinja code](../examples/tutorial/code_jinja.pdl)),
([shell command](../examples/tutorial/code_command.pdl)),
([PDL code](../examples/tutorial/code_pdl.pdl)).

## Calling REST APIs

PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app ([file](../examples/tutorial/programs/weather.pdl)):

```yaml
--8<-- "./examples/tutorial/programs/weather.pdl"
```

In this program, we first define a query about the weather in some location (assigned to variable `QUERY`). The next block is a call to a Granite model with few-shot examples to extract the location, which we assign to variable `LOCATION`. The next block makes an API call with Python (mocked in this example). Here the `LOCATION` is appended to the `url`. The result is a JSON object, which may be hard to interpret for a human user. So we make a final call to an LLM to interpret the JSON in terms of weather. Notice that many blocks have `contribute` set to `[]` to hide intermediate results.

## Data Block

PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the [Overview](./tutorial.md#overview) section). The `data` block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file ([file](../examples/tutorial/programs/code-json.pdl)).

```yaml
--8<-- "./examples/tutorial/programs/code-json.pdl"
```

Notice that in the `data` block the values are interpreted as Jinja expressions. If values need to be PDL programs to be interpreted, then you need to use
the `object` block instead (see [this section](#specifying-data)).

In the example above, the expressions inside the `data` block are interpreted, but in some cases it may be useful not to interpret the values in a `data` block.
The `raw` field can be used to turn off the interpreter inside a `data` block. For example, consider the ([file](../examples/tutorial/data_block_raw.pdl)):

```yaml
--8<-- "./examples/tutorial/data_block_raw.pdl"
```

The result of this program is the JSON object:

```
{
  "name": "${ name }",
  "phone": "${ phone }"
}
```

where the values of `name` and `phone` have been left uninterpreted.

## Import Block

PDL allows programs to be defined over multiple files. The `import` block allows one file to incorporate another, as shown in the
following [example](../examples/tutorial/import.pdl):

```yaml
--8<-- "./examples/tutorial/import.pdl"
```

which imports the following [file](../examples/tutorial/import_lib.pdl):

The `import` block means that the PDL code at that file is executed and its scope is assigned to the variable defined in that block. So all the `defs` in the imported file are made available via that variable. This feature allows reuse of common templates and patterns and to build libraries. Notice that relative paths are relative to the containing file.

## Prompt Library

A prompt library is included in the `contrib/` directory. These modules define some common patterns such Chain-of-Thought, ReAct, and ReWOO. Example usage can be found in `examples/prompt_library`. Note that `import` blocks resolve file paths relative to the PDL file being executed.

### Chain-of-Thought

In the `GSM8K` example below, we import `CoT` [(Wei et al., 2022)](https://arxiv.org/abs/2201.11903) and define our model, demonstrations, and the question. Demonstrations are a list of dictionaries with `question`, `reasoning`, and `answer` keys. The `cot.chain_of_thought` function returns a dictionary with an `answer` key.

``` yaml title="examples/prompt_library/gsm8k_cot.pdl"
--8<-- "./examples/prompt_library/gsm8k_cot.pdl"
```

Executing this example produces the following background context (model response highlighted):

``` text hl_lines="27-31"
Answer the questions to the best of your abilities.

Question: Noah charges $60 for a large painting and $30 for a small painting.
Last month he sold eight large paintings and four small paintings.
If he sold twice as much this month, how much is his sales for this month?
Answer: Let's think step by step. He sold 8 large paintings and 4 small paintings last month.
He sold twice as many this month.
8 large paintings x $60 = << 8*60= 480 >> 480
4 small paintings x $30 = << 4*30= 120 >> 120
So he sold << 480+120= 600 >> 600 paintings last month.
Therefore he sold << 600*2= 1200 >> this month.
The answer is $1200

Question: Noah charges $30 for a large vases and $10 for a small vases.
Last month he sold five large vases and three small vases.
If he sold three times as much this month, how much is his sales for this month?
Answer: Let's think step by step. He sold 5 large vases and 3 small vases last month.
He sold three times as many this month.
5 large vases x $30 = << 5*30= 150 >> 150
3 small vases x $10 = << 3*10= 30 >> 30
So he sold << 150+30= 180 >> 180 vases last month.
Therefore he sold << 180*3= 540 >> this month.
The answer is $540

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Answer: Let's think step by step.
1. Jacob earns $6 per hour.
2. Jake earns thrice what Jacob does, so Jake earns 3 * $6 = $18 per hour.
3. Jake works 8 hours a day, so he earns $18 * 8 = $144 per day.
4. Jake works 5 days, so he earns $144 * 5 = $720 in 5 days.
The answer is $720.
```

The final result is:

``` text
Answer the questions to the best of your abilities.

Result: {'answer': '1. Jacob earns $6 per hour.\n2. Jake earns thrice what Jacob does, so Jake earns 3 * $6 = $18 per hour.\n3. Jake works 8 hours a day, so he earns $18 * 8 = $144 per day.\n4. Jake works 5 days, so he earns $144 * 5 = $720 in 5 days.\nThe answer is $720.'}
```

### ReAct

In the ReAct [(Yao et al., 2022)](https://arxiv.org/abs/2210.03629) example below, we import the ReAct library and provide tools from `tools.pdl`. Demonstrations consist of agent trajectories represented as a list of lists with key names `question` or `task`, `thought`, `action`, and `observation`. Actions follow this pattern: `{"name": "tool_name", "arguments": {"arg1": "..."}}`. The function returns a dictionary with a `answer` key, containing the answer.

``` yaml title="examples/prompt_library/gsm8k_react.pdl"
--8<-- "./examples/prompt_library/gsm8k_react.pdl"
```

Produces background context (model responses highlighted):

``` text hl_lines="22 23 25 26 28 29 31 32"
Question: Noah charges $60 for a large painting and $30 for a small painting.  Last month he sold eight large paintings and four small paintings.  If he sold twice as much this month, how much is his sales for this month?
Tho: He sold 8 large paintings and 4 small paintings last month.  He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30)
Act: {"name": "calculator", "arguments": {"expr": "8*60+4*30"}}
Obs: 600
Tho: He sold twice as many paintings this month, therefore I need to calculate 600*2.
Act: {"name": "calculator", "arguments": {"expr": "600*2"}}
Obs: 1200
Tho: He sold $1200 this month.
Act: {"name": "finish", "arguments": {"answer": "$1200"}}

Question: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
Tho: I need to calculate the difference in age between Teresa and Morio.
Act: {"name": "calculator", "arguments": {"expr": "71-59"}}
Obs: 12
Tho: I need to calculate how old Teresa is when their daughter is born.
Act: {"name": "calculator", "arguments": {"expr": "38-12"}}
Obs: 26
Tho: Teresa was 26 when she gave birth to Michiko.
Act: {"name": "finish", "arguments": {"answer": "26"}}

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Tho: Jacob earns $6 per hour. Jake earns thrice as much.
Act: {"name": "calculator", "arguments": {"expr": "6*3"}}
Obs: 18
Tho: Jake earns $18 per hour.
Act: {"name": "calculator", "arguments": {"expr": "18*8"}}
Obs: 144
Tho: Jake earns $144 in a day.
Act: {"name": "calculator", "arguments": {"expr": "144*5"}}
Obs: 720
Tho: Jake earns $720 in 5 days.
Act: {"name": "finish", "arguments": {"answer": "$720"}}
```

The final result is:

``` text
Result: {'answer': '$720'}
```

#### Tools

Tools allow the agentic patterns to call PDL functions. The tools are defined in two parts. The `tools` object is a dictionary of tool names to PDL functions. The `tool_schema` is a JSON schema, represented as a `data` block consisting of list of dictionaries, that describes the tools to the LLM. The `tools` library contains a calculator and a Wikipedia search tool by default. There are a few requirements to consider when using tools.

1. You must have a `finish` tool in the _schema_. You do not need a PDL function for this, but the LLM needs to be aware of the `finish` tool so the `ReAct` loop can end.
2. A PDL tool function must accept only `arguments: obj` as parameters.

Here we define our tools in PDL, with an example using a `python` block. Note the use of `arguments: obj`.

``` yaml
tools:
  object:
    hello:
      function:
        arguments: obj
      return:
        lang: python
        code: |
          result = None
          def main(name: str, *args, **kwargs) -> str:
            return f"hello {name}"
          result = main(**arguments)

    another_function: ...
```

Once we have defined our PDL functions, we describe them in the `tool_schema`.

``` yaml
tool_schema:
  data:
    - name: hello
      description: Hello function, returns "hello <name>"
      parameters:
        type: object
        properties:
          name:
            type: string
            description: Name to greet
        required:
          - name

    - name: finish
      description: Respond with the answer
      parameters:
        type: object
        properties:
          answer:
            type: str
            description: The answer
          required:
            - answer
```

Another useful thing to remember is that data blocks are templated in PDL. For example, this is valid:

``` yaml
finish_action:
  data:
    name: finish
    description: Respond with the answer
    parameters:
      type: object
      properties:
        answer:
          type: str
          description: The answer
        required:
          - answer

tool_schema:
  data:
    - name: calculator
      description: Calculator function
      parameters:
        type: object
        properties:
          expr:
            type: string
            description: Arithmetic expression to calculate
        required:
          - expr
    - ${ finish_action }
```

In fact, you can reuse the `finish_action` from `tools` in your own tool schemas:

``` yaml
defs:
  tools:
    import: ../../contrib/prompt_library/tools
  my_tool_schema:
    data:
      - ${ tools.finish_action }
      - ...
```

Additionally, there is a helper function `filter_tools_by_name` in `tools`, that given a JSON tools schema and a list of tool names, returns a schema with only those tools in it. This is useful if you have defined many tools, but don't need all of them for a particular task.

``` yaml
call: ${ tools.filter_tools_by_name }
args:
  tools: ${ tools.tool_schema }
  tool_names: ['calculator', 'finish']
```

### ReWOO

The tools can be reused for ReWOO [(Xu et al., 2023)](https://arxiv.org/abs/2305.18323). Demonstrations follow a similar pattern as for ReAct, except that ReWOO does not use a `finish` tool/action, nor does it include `observation`, and thus should be omitted from the demonstrations. The argument `show_plans` is used for displaying the extracted ReWOO plans in the background context for debugging purposes i.e. `--stream context`. The function returns a dictionary with a `answer` key, containing the answer.

``` yaml title="examples/prompt_library/gsm8k_rewoo.pdl"
--8<-- "./examples/prompt_library/gsm8k_rewoo.pdl"
```

This results in the following background context:

``` text hl_lines="18 28-30"
For the following task, make plans that can solve the problem step by step. For each plan, indicate which external tool together with tool input to retrieve evidence. You can store the evidence into a variable #E that can be called by later tools. (Plan, #E1, Plan, #E2, Plan, ...)

Tools can be one of the following:
[{'type': 'function', 'function': {'name': 'calculator', 'description': 'Calculator function', 'parameters': {'type': 'object', 'properties': {'expr': {'type': 'string', 'description': 'Arithmetic expression to calculate'}}, 'required': ['expr']}}}]

Task: Noah charges $60 for a large painting and $30 for a small painting. Last month he sold eight large paintings and four small paintings. If he sold twice as much this month, how much is his sales for this month?
Plan: He sold 8 large paintings and 4 small paintings last month. He sold twice as many this month. I need to calculate (8 large paintings x $60 + 4 small paintings x $30) #E1 = {"name": "calculator", "arguments": {"expr": "8*60+4*30"}}
Plan: He sold twice as many paintings this month, therefore I need to calculate 600*2. #E2 = {"name": "calculator", "arguments": {"expr": "600*2"}}

Task: Teresa is 59 and her husband Morio is 71 years old. Their daughter, Michiko was born when Morio was 38. How old was Teresa when she gave birth to Michiko?
Plan: I need to calculate the difference in age between Teresa and Morio. #E1 = {"name": "calculator", "arguments": {"expr": "71-59"}}
Plan: I need to calculate how old Teresa is when their daughter is born. #E2 = {"name": "calculator", "arguments": {"expr": "38-12"}}

Begin!
Describe your plans with rich details. Each Plan should be followed by only one #E.

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
[["First, I need to calculate Jake's hourly wage. Since Jake earns thrice what Jacob does, and Jacob earns $6 per hour, Jake earns 3 * $6 = $18 per hour.", "#E1", "{\"name\": \"calculator\", \"arguments\": {\"expr\": \"6*3\"}}"]]

Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information.

Plan: First, I need to calculate Jake's hourly wage. Since Jake earns thrice what Jacob does, and Jacob earns $6 per hour, Jake earns 3 * $6 = $18 per hour.
Evidence: 18

Now solve the question or task according to provided Evidence above. Respond with the answer directly with no extra words.

Question: Jake earns thrice what Jacob does. If Jacob earns $6 per hour, how much does Jake earn in 5 days working 8 hours a day?
Response: Jake earns $18 per hour. In 5 days, working 8 hours a day, Jake works 5 * 8 = 40 hours. Therefore, Jake earns 40 * $18 = $720 in 5 days.

Answer: $720
```

The final result of this example is

``` text
Result: {"answer": "Jake earns $18 per hour. In 5 days, working 8 hours a day, Jake works 5 * 8 = 40 hours. Therefore, Jake earns 40 * $18 = $720 in 5 days.\n\nAnswer: $720"}
```

## Conditionals and Loops

PDL supports conditionals and loops as illustrated in the following example ([file](../examples/tutorial/programs/chatbot.pdl)), which implements a chatbot.

```yaml
--8<-- "./examples/tutorial/programs/chatbot.pdl"
```

The first block prompts the user for a query, and this is contributed to the background context. The next
block is a `repeat-until`, which repeats the contained `text` block until the condition in the `until` becomes
true. The field `repeat` can contain a string, or a block, or a list. If it contains a list, then the list must be a `text`,
`array` or `lastOf` (which means that all the blocks in the list are executed and the result of the body is that of the last block).

The example also shows the use of an `if-then-else` block. The `if` field contains a condition, the `then` field
can also contain either a string, or a block, or a list (and similarly for `else`).

The chatbot keeps looping by making a call to a model, asking the user if the generated text is a good answer,
and asking `why not?` if the answer (stored in variable `eval`) is `no`. The loop ends when `eval` becomes `yes`. This is specified with a Jinja expression on line 18.

Notice that the `repeat` and `then` blocks are followed by `text`. This is because of the semantics of lists in PDL. If we want to aggregate the result by stringifying every element in the list and collating them together, then we need the keyword `text` to precede a list. The number of iterations of a loop can be bounded by adding a `max_iterations` field.

The way that the result of each iteration is collated with other iterations can be customized in PDL using
the `join` feature (see the following section).

Another simple example of using an `if` statement is [this](../examples/tutorial/if.pdl).

### For Loops

PDL also offers `for` loops over lists.
The following [example](../examples/tutorial/for.pdl) stringifies and outputs each number.

```yaml
--8<-- "./examples/tutorial/for.pdl"
```

This program outputs:

```
1234
```

To output a number of each line, we can specify which string to use to join the results.

```yaml
--8<-- "./examples/tutorial/for_with.pdl"
```

```
1
2
3
4
```

To create an array as a result of iteration, we would write:

```yaml
--8<-- "./examples/tutorial/for_array.pdl"
```

which outputs the following list:

```
[1, 2, 3, 4]
```

To retain only the result of the last iteration of the loop, we would write:

```yaml
--8<-- "./examples/tutorial/for_lastOf.pdl"
```

which outputs:

```
4
```

When `join` is not specified, the collation defaults to

```
join:
  as: text
  with: ""
```

meaning that result of each iteration is stringified and concatenated with that of other iterations. When using `with`,
`as: text` can be elided.

Note that `join` can be added to any looping construct (`repeat`) not just `for` loops.

The `for` loop construct also allows iterating over 2 or more lists of the same length simultaneously:

```yaml
--8<-- "./examples/tutorial/for_multiple_lists.pdl"
```

This results in the following output:

```
Bob's number is 1
Carol's number is 2
David's number is 3
Ernest's number is 4
```

The loop constructs also allow to build an object:

```yaml
--8<-- "./examples/tutorial/for_object.pdl"
```

This results in the following output:

```
{"Bob": 1, "Carol": 2, "David": 3, "Ernest": 4}
```

### While Loop

The following example shows a while loop in PDL:

```yaml
--8<-- "./examples/tutorial/while.pdl"
```

The `while` field indicates the looping condition and `repeat` contains the body of the loop.

Notice that `for`, `while`, `until`, and `maxIterations` can all be combined in the same `repeat` block. The loop exits
as soon as one of the exit conditions is satisfied:

```yaml
--8<-- "./examples/tutorial/repeat.pdl"
```

### Match block

PDL provides a match block for convenience.
Consider the [example](../examples/tutorial/programs/demo-hallucination.pdl). This shows retrieved RAG documents
that are then submitted with a query to a RAG Granite model.
The output contains an answer to the query together with hallucination
score and possibly a citation.

To obtain and install the Granite model locally follow these
[instructions](../examples/intrinsics/README.md).

The end of this program contains a match block:

```yaml
...
  The answer is: ${ out[0].sentence }
- match: ${out[0].meta.hallucination_level}
  with:
  - case: "high"
    then: Totally hallucinating, sorry!
  - case: "low"
    if: ${ out[0].meta.citation }
    then: |
      I am not hallucinating, promise!
      The citation is: ${ out[0].meta.citation.snippet }
  - then: Not sure if I am hallucinating...
```

The `match` field indicates an expression to match on. The cases follow the `with` field. Additional conditions can be indicated as shown in the second case.

## Roles and Chat Templates

Consider again the chatbot example ([file](../examples/tutorial/programs/chatbot.pdl)). By default blocks have role `user`, except for model call blocks, which have role `assistant`.
If we write roles explicitly for the chatbot, we obtain:

```yaml
description: chatbot
text:
- read:
  message: "What is your query?\n"
  contribute: [context]
- repeat:
    text:
    - model: replicate/ibm-granite/granite-3.1-8b-instruct
      role: assistant
    - read:
      def: eval
      message: "\nIs this a good answer[yes/no]?\n"
      contribute: []
    - if: ${ eval == 'no' }
      then:
        text:
        - read:
          message: "Why not?\n"
  until: ${ eval == 'yes'}
role: user
```

In PDL, any block can be adorned with a `role` field indicating the role for that block. These are high-level annotations
that help to make programs more portable across different models. If the role of a block is not specified (except for model blocks that have `assistant` role),
then the role is inherited from the surrounding block. So in the above example, we only need to specify `role: user` at the top level (this is the default, so it doesn't
need to be specified explicitly).

PDL takes care of applying appropriate chat templates (done either in LiteLLM or at the server side).

The prompt that is actually submitted to the first model call (with query `What is APR?`) is the following:

```
<|start_of_role|>user<|end_of_role|>What is APR?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>
```

To change the template that is applied, you can specify it as a parameter of the model call:

```yaml
model: replicate/ibm-granite/granite-3.1-8b-instruct
parameters:
  roles:
    system:
       pre_message: <insert text here>
       post_message: <insert text here>
    user:
       pre_message: <insert text here>
       post_message: <insert text here>
    assistant:
       pre_message: <insert text here>
       post_message: <insert text here>
```

## Type Checking

Consider the following PDL program ([file](../examples/tutorial/type_checking.pdl)). It first reads the data
found [here](../examples/tutorial/type_checking_data.yaml) to form few-shot examples. These demonstrations show how to create
some JSON data.

```yaml
--8<-- "./examples/tutorial/type_checking.pdl"
```

Upon reading the data we use a parser to parse it into a YAML. The `spec` field indicates the expected type for the
data, which is an object with 2 fields: `questions` and `answers` that are a list of string and a list of objects,
respectively. When the interpreter is executed, it checks this type dynamically and throws errors if necessary.

Similarly, the output of the model call is parsed as YAML, and the `spec` indicates that we expect an object with
2 fields: `name` of type string, and `age` of type integer.

When we run this program, we obtain the output:

```
{'name': 'John', 'age': '30'}
type_checking.pdl:9 - Type errors during spec checking:
type_checking.pdl:9 - twentyfive should be of type <class 'int'>
```

Notice that since we asked the age to be produced in letters, we got a string back and this causes a type error indicated above.

In general, `spec` definitions can be a subset of JSON schema, or use a shorthand notation as illustrated by
the examples below:

- `bool`: boolean
- `str`: string
- `int`: integer
- `float`: float
- `{str: {pattern: '^[A-Za-z][A-Za-z0-9_]*$'}}`: a string satisfying the indicated pattern
- `{float: {minimum: 0, exclusiveMaximum: 1}}`: a float satisfying the indicated constraints
- `{list: int}`: a list of integers
- `[int]`: a list of integers
- `{list: {int: {minimum: 0}}}`: a list of integers satisfying the indicated constraints
- `[{int: {minimum: 0}}]`: same as above
- `{list: {minItems: 1, int: {}}}`, a list satisfying the indicated constraints
- `{obj: {latitude: float, longitude: float}}`: an object with fields `latitude` and `longitude`
- `{latitude: float, longitude: float}`: same as above
- `{obj: {question: str, answer: str, context: {optional: str}}}`: an object with an optional field
- `{question: str, answer: str, context: {optional: str}}`: same as above
- `{list: {obj: {question: str, answer: str}}}`: a list of objects
- `[{question: str, answer: str}]`: same as above
- `{enum: [red, green, blue]}`: an enumeration

Another example of type checking a list can be found [here](../examples/tutorial/type_list.pdl).

## Structured Decoding

When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields `guided_json` and `response_format` are added automatically by the interpreter with a JSON Schema value obtained from the type. Models on platforms that support structured decoding will then use this to generate JSON of the correct format.

The following [program](../examples/tutorial/structured-decoding.pdl):

```yaml
--8<-- "./examples/tutorial/structured_decoding.pdl"
```

produces the output:

```

What is the color of the sky?
{'color': 'blue'}
```

## Python SDK

See examples of PDL being called programmatically in Python
[here](../examples/sdk).

For a more sophisticated example, see [here](../examples/callback).

## Debugging PDL Programs

We highly recommend to edit PDL programs using an editor that support YAML with JSON Schema validation. For example, you can use VSCode with the [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) and configure it to use the [PDL schema](../src/pdl/pdl-schema.json). The PDL repository has been configured so that every `*.pdl` file is associated with the PDL grammar JSONSchema (see [settings](../.vscode/settings.json)).
This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. The PDL interpreter also provides similar error messages. To make sure that the schema is associated with your PDL files, be sure that `PDL Schemas` appear at the bottom right of your VSCode window, or on top of the editor window.

To run the GUI, go to the `pdl-live-react` directory and run to following commands:
```
npm i
npm start
```

To generate a trace for use in the GUI:

```bash
pdl --trace <file.json> <my-example.pdl>
```

This is similar to a spreadsheet for tabular data, where data is in the forefront and the user can inspect the formula that generates the data in each cell. In the Live Document, cells are not uniform but can take arbitrary extents. Clicking on them similarly reveals the part of the code that produced them.

Finally, PDL includes experimental support for gathering trace telemetry.  This can
be used for debugging or performance analysis, and to see the shape of prompts sent by LiteLLM to models.

For more information see [here](./telemetry.md).

## Using Ollama models

1. Install Ollama e.g., `brew install --cask ollama`
2. Run a model e.g., `ollama run granite-code:8b`. See [the Ollama library for more models](https://ollama.com/library/granite-code/tags)
3. An OpenAI style server is running locally at [http://localhost:11434/](http://localhost:11434/), see [the Ollama blog](https://ollama.com/blog/openai-compatibility) for more details.

Example:

```
text:
- Hello,
- model: ollama_chat/granite-code:8b
  parameters:
    stop:
    - '!'
    decoding_method: greedy
```

If you want to use an external Ollama instance, the env variable `OLLAMA_API_BASE` should be defined, by default is `http://localhost:11434`.

Alternatively, one could also use Ollama's OpenAI-style endpoint using the `openai/` prefix instead of `ollama_chat/`. In this case, set the `OPENAI_API_BASE`, `OPENAI_API_KEY`, and `OPENAI_ORGANIZATION` (if necessary) environment variables. If you were using the official OpenAI API, you would only have to set the api key and possibly the organization. For local use e.g., using Ollama, this could look like so:

```bash
export OPENAI_API_BASE=http://localhost:11434/v1
export OPENAI_API_KEY=ollama # required, but unused
export OPENAI_ORGANIZATION=ollama # not required

pdl <...>
```

## Strings In Yaml

Multiline strings are commonly used when writing PDL programs. There are two types of formats that YAML supports for strings: block scalar and flow scalar formats. Scalars are what YAML calls basic values like numbers or strings, as opposed to complex types like arrays or objects. Block scalars have more control over how they are interpreted, whereas flow scalars have more limited escaping support. (Explanation here is thanks to [Wolfgang Faust](https://yaml-multiline.info/))

### Block Scalars

**Block Style Indicator**: The block style indicates how newlines inside the block should behave. If you would like them to be **kept** as newlines, use the literal style, indicated by a pipe `|`. Note that without a chomping indicator, described next, only the *last* newline is kept.

PDL:

```
text:
  - |
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."
```

Output:

```
Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.
End.
```

If instead you want them to be replaced by spaces, use the folded style, indicated by a right angle bracket `>`. To get a newline using the folded style, leave a blank line by putting *two* newlines in. Lines with extra indentation are also not folded.

PDL:

```
text:
  - >
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."
```

Output:

```
Several lines of text, with some "quotes" of various 'types', and also a blank line:
and some text with
    extra indentation
on the next line, plus another line at the end.
End.
```

**Block Chomping Indicator**: The chomping indicator controls what should happen with newlines at the end of the string. The default, clip, puts a single newline at the end of the string. To remove all newlines, strip them by putting a minus sign `-` after the style indicator. Both clip and strip ignore how many newlines are actually at the end of the block; to keep them all put a plus sign `+` after the style indicator.

PDL:

```
text:
  - |-
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."
```

Output:

```
Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.End.
```

PDL:

```
text:
  - |+
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "End."
```

Output:

```
Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.


End.
```

If you don't have enough newline characters using the above methods, you can always add more like so:

```
text:
  - |-
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line,
    plus another line at the end.


  - "\n\n\n\n"
  - "End."
```

Output:

```
Several lines of text,
with some "quotes" of various 'types',
and also a blank line:

and some text with
    extra indentation
on the next line,
plus another line at the end.



End.
```

**Indentation Indicator**: Ordinarily, the number of spaces you're using to indent a block will be automatically guessed from its first line. You may need a block indentation indicator if the first line of the block starts with extra spaces. In this case, simply put the number of spaces used for indentation (between 1 and 9) at the end of the header.

PDL:

```
text:
  - |1
    Several lines of text,
    with some "quotes" of various 'types',
    and also a blank line:

    and some text with
        extra indentation
    on the next line.
```

Output:

```
 Several lines of text,
 with some "quotes" of various 'types',
 and also a blank line:

 and some text with
     extra indentation
 on the next line.
```

### Flow Scalars

**Single-quoted**:

PDL:

```
text: 'Several lines of text,
  containing ''single quotes''. Escapes (like \n) don''t do anything.

  Newlines can be added by leaving a blank line.
    Leading whitespace on lines is ignored.'
```

Output:

```
Several lines of text, containing 'single quotes'. Escapes (like \n) don't do anything.
Newlines can be added by leaving a blank line. Leading whitespace on lines is ignored.
```

**Double-quoted**:

PDL:

```
text: "Several lines of text,
  containing \"double quotes\". Escapes (like \\n) work.\nIn addition,
  newlines can be esc\
  aped to prevent them from being converted to a space.

  Newlines can also be added by leaving a blank line.
    Leading whitespace on lines is ignored."
```

Output:

```
Several lines of text, containing "double quotes". Escapes (like \n) work.
In addition, newlines can be escaped to prevent them from being converted to a space.
Newlines can also be added by leaving a blank line. Leading whitespace on lines is ignored.
```

**Plain**:

PDL:

```
text: Several lines of text,
  with some "quotes" of various 'types'.
  Escapes (like \n) don't do anything.

  Newlines can be added by leaving a blank line.
    Additional leading whitespace is ignored.
```

Output:

```
Several lines of text, with some "quotes" of various 'types'. Escapes (like \n) don't do anything.
Newlines can be added by leaving a blank line. Additional leading whitespace is ignored.
```
