Serialisation

I need to be able to save the setting into templates for the user to load, but first I need to finish off the exta options.

Extra Options

At present this is just a label placeholder. So I will get the agent to replace this.

change the more_options_label into a QLineEdit. This will allow the user to pass extra flags verbatim to the launch command line. 

That was nice and quick and worked first time, see AgentChat29.md

Saving to disk

I am thinking of using json for the serialisation of the user templates but it would be interesting to see what the agent thinks.

I have this in the TODO.md

## Phase 4 : Configuration save and load

[ ] add the ability to save configuration to a json file
[ ] add the ability to load configuration from a json file
[ ] Use QSettings to save/load the last setup

[@TODO.md (206:210)](file:///Volumes/teaching/Code/LLamaLauncher/TODO.md#L206:210) [@main.py](file:///Volumes/teaching/Code/LLamaLauncher/main.py) need to implement the saving and loading of the current setting into a format that the user can use to load "template setting files". What file format would you suggest to save these so the user can re-load settings.

It has done a good analysis of the project and decided to use json as the format.

Proposing a schema

{
  "version": "1.0",
  "files": { ... },
  "server": { ... },
  "sampling": { temperature: {enabled, value}, ... },
  "performance": { gpu_layers: {enabled, value}, ... },
  "advanced": { draft_model: {enabled, path}, ... },
  "context_size": "8K",
  "more_options": "",
  "no_mmproj_offload": false
}

Even though I am in “Ask” mode it seems to be getting on with things, I need to investigate this more.

It has decided to do the following

Now let me implement this. I'll add:
1. A **File** menu with Save/Save As/Load actions
2. `_save_config()` and `_load_config()` methods
3. **QSettings** for last-session persistence (auto-saved on close, auto-loaded on init)

Again it has stalled (could be the Ask mode perhaps?) I will create a new thread and get it to implement each bit.

Now it is implementing a worktree to do the work! this is so inconsistent, let’s see what it generates.

It seems to have generated a load of code but has go stuck in a rut again AgentChat30.md

We now have a save and load screen but it errors when running and I’m out of context.

./main.py
js: [ToolsStore] Failed to fetch built-in tools: Error: File Not Found
Traceback (most recent call last):
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 188, in _save_config
    self._save_config_as()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 201, in _save_config_as
    self._write_config_file(file_path)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 210, in _write_config_file
    config = self._collect_config()
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 228, in _collect_config
    "model_path": self.model_path_edit.getProperty("fullPath", ""),
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'PySide6.QtWidgets.QLineEdit' object has no attribute 'getProperty'. Did you mean: 'setProperty'?
Error calling Python override of QMainWindow::closeEvent(): Traceback (most recent call last):
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 489, in closeEvent
    self._save_last_session()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Volumes/teaching/Code/LLamaLauncher/./main.py", line 495, in _save_last_session
    settings.setValue("lastModelPath", self.model_path_edit.getProperty("fullPath", ""))
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'PySide6.QtWidgets.QLineEdit' object has no attribute 'getProperty'. Did you mean: 'setProperty'?

We’ve had this error before, seems like it guesses the Qt coding standard rather than know it. (Qt states that setters are set[Thing] and getters are just Thing() )

Seems to have fixed it and I can save the settings.

{
  "version": "1.0",
  "files": {
    "model_path": "/Volumes/teaching/AI_Models/Llama-3.2-1B-Instruct-Q4_K_M.gguf",
    "mmproj_path": "",
    "draft_model_path": "",
    "json_schema_path": ""
  },
  "server": {
    "host": "127.0.0.1",
    "port": 8080,
    "api_key": "12345"
  },
  "sampling": {
    "temperature": {
      "enabled": true,
      "value": 0.6
    },
    "top_p": {
      "enabled": true,
      "value": 0.9
    },
    "top_k": {
      "enabled": true,
      "value": 20
    },
    "min_p": {
      "enabled": false,
      "value": 0.05
    },
    "typical_p": {
      "enabled": false,
      "value": 1.0
    },
    "repeat_penalty": {
      "enabled": true,
      "value": 1.1
    },
    "repeat_last_n": {
      "enabled": true,
      "value": 64
    },
    "presence_penalty": {
      "enabled": false,
      "value": 0.0
    },
    "frequency_penalty": {
      "enabled": false,
      "value": 0.0
    },
    "mirostat": {
      "enabled": false,
      "value": 1
    },
    "mirostat_lr": {
      "enabled": false,
      "value": 0.1
    },
    "mirostat_ent": {
      "enabled": false,
      "value": 5.0
    }
  },
  "performance": {
    "gpu_layers": {
      "enabled": true,
      "value": 99
    },
    "threads": {
      "enabled": true,
      "value": 4
    },
    "threads_batch": {
      "enabled": false,
      "value": 0
    },
    "batch_size": {
      "enabled": false,
      "value": 512
    },
    "ubatch_size": {
      "enabled": false,
      "value": 512
    },
    "n_predict": {
      "enabled": false,
      "value": 2048
    },
    "parallel": {
      "enabled": false,
      "value": 4
    },
    "flash_attn": "auto",
    "cache_type_k": {
      "enabled": false,
      "value": "f16"
    },
    "cache_type_v": {
      "enabled": false,
      "value": "f16"
    },
    "mmap": true,
    "mlock": false,
    "cont_batching": false
  },
  "advanced": {
    "draft_model": {
      "enabled": false,
      "path": ""
    },
    "spec_draft_n_max": {
      "enabled": false,
      "value": 5
    },
    "seed": {
      "enabled": false,
      "value": 42
    },
    "grammar": {
      "enabled": false,
      "path": ""
    },
    "json_schema": {
      "enabled": false,
      "path": ""
    },
    "rope_scaling": {
      "enabled": false,
      "value": "none"
    },
    "rope_freq_base": {
      "enabled": false,
      "value": 1.0
    },
    "rope_freq_scale": {
      "enabled": false,
      "value": 1.0
    }
  },
  "context_size": 8192,
  "more_options": "",
  "no_mmproj_offload": false
}

First thing I notices is the core components from the QSetting are persisting between runs now which is nice. The file can be re-loaded so we have a way of saving the templates.

Code Review

Time to have a quick look at the code now, we are nearly at 900 lines with no tests which is not ideal.

There are some function > 100 lines long with the launch one being 171 lines with lots of embeded if statements.

This is one of the main issues with this style of coding, you genreate hard to maintain code which can be quite messy, the other problem is the fact I have not be forcing a testing regime which makes the re-factor so much harder to do with confidence.

Think it is time to start thinking about adding tests, before doing some code tidy ups.