What is Enkaidu?#

Agentic#

At its simplest, Enkaidu is a user input / AI inference loop.

  while prompt = gets do
    while resp = llm(prompt) do
      break unless resp.is_a? Call
      prompt = call(resp)
    end
    puts resp
  end

With the right model this loop can drive a cycle of inference and tool calling to perform your request. This is the essence of agentic.

Context windows#

Local AI models are small, and they have context windows limited by the available memory. In contract with commercial cloud-hosted LLMs with hundres of thousands of tokens in the context window, local models have tens of thousands at best. This is before the system prompt and tool definitions. Every subsequent user request can quickly fill up the context window.

Forkable Sessions#

Enkaidu supports the ability to “fork” the current session, letting you get started on some activity and then derive a new session to perform a task and then resume the original session within retaining all that context.

Use /session push and do some work and return with a /session pop. This “nesting” lets us use the context window we have to the best of its effort.

There is also /session pop_and_take which lets you bring back the last response in the nested session.

Dynamic Tools#

Enkaidu supports the ability to load and unload tools, either individually or as entire toolsets, before and after prompting the LLM. This can help in two ways:

1. Tool definitions take of context window space, so enabling tools when you want them can make a big difference.
2. LLMs, especially smaller local ones, can get overwhelmed by too many tools, especially when there may seem to be overlap between tools.

Use /tool ... or /toolset ... commands.

Macros#

Enkaidu can “play back” macros that you define using prompts and slash commands. This lets you define common tasks and also dynamically control the session for those tasks.

Consider a macro that forks the session, does some work, and then returns to the current session with some content.

How about a macro that loads a tool or toolset to perform a task and then unloads them afterwards to keep the context window usage under control?

Use the macros: section in your configuration file, or in your profile folder, to define macros, and then the !<MACRONAME> to play it.

And yes, macros can “call” other macros!

Dynamic Tools#

Enkaidu supports the ability to load and unload tools, either individually or as entire toolsets, before and after prompting the LLM. This can help in two ways:

1. Tool definitions take of context window space, so enabling tools when you want them can make a big difference.
2. LLMs, especially smaller local ones, can get overwhelmed by too many tools, especially when there may seem to be overlap between tools.