Vector Suppport for locally running Laguage Models embedding support

[Agredo]

I wanted to bring this up again because no discussion has developed since 2023 and we are making huge leaps in directions where locally running language models are becoming more common.

#2364

It is currently very hard to find a locally running database that supports vector storage queries.
Actually, storage is not the problem.

For example, storing an n-dimensional vector in SQLite is not the problem.

In my case, I create an embedding with https://github.com/dotnet/smartcomponents. But the problem is the search. There is no native way to use cosine similarity or nearest neighbor for vectors stored in SQLite without using an extension like sqlite-vec (https://github.com/asg017/sqlite-vec). So it is necessary to query all vecotors and doing the similarity search manually

In .Net there are already some implementations for vector storage in memory like https://github.com/Build5Nines/SharpVector but nothing locally persistent and here a native implementation for LiteDB would be great!

Initially it would be enough for me to just add a ByteArray or a FloatArray and have similarity and NN search function on it.

[JKamsker]

Features are currently mostly submitted by the community. If you want to open a pr for that, i would be happy to accept it :)