If you have ever worked with
#GaussianProcesses, you know that designing kernels is not that simple. It requires extensive prior knowledge and effort.
Over the past couple of days, I have been toying with the idea of getting
#GPT4 to write kernel code for you after providing some description of your function. This is trying to automate GP model fitting by going from natural language text to kernel code that we could use for training a GP.
The idea seems to work (more or less). I just open-sourced the code that does this. For now, this code uses sci-kit-learn, but I don't see why it wouldn't work with GPytorch. What this code does is the following:
1. It takes from you a prompt like the one in the figure below. It will then ask GPT from
@OpenAI to generate for you a GP kernel.
2. The generated kernel is then checked to determine its validity. Also (thanks to
@alex_maraval ), I added a check to make sure the exec is not doing anything nasty:
3. Once all this is checked, it goes to the model fitting part, where the GP model is fit but using the
#GPT4 generated code:
4. After the fit, we predict a test/validation data set and record the marginal, which is then reflected to the LLM with the marginals and codes generated previously to generate a new kernel code that is better than before.
5. The process repeats for a maximum number of iterations.
The method seems to work more or less. Of course, it is just phase one. Next, I plan to improve the following:
1. Write a data loader to upload your data and add descriptions.
2. Make a website for you to use.
3. Using something other than GPT4o, maybe something open-source
#llama 3. I don't have the resources to host it personally, though.
I hope you like it, and please feel free to contribute. I really would love to get everyone using
#GPs again, and I am hoping with
#LLMs, many things can be automated. This is but step 1 :D
If you find the code useful, please star the repo :)
#AI #MachineLearning
