Generate better garbage data

[JasonLovesDoggo]

Looking for suggestions into how

[JasonLovesDoggo]

https://arxiv.org/pdf/2305.17493v2 - Perhaps use the "jackrabbit" data if it can be found?

[JasonLovesDoggo]

This should probably be written out as a separate module as it can be useful for other projects.

Some generation options include:

• Code
• Language (e.g. full proper words, just in an order that doesn't make sense)

[JasonLovesDoggo]

constrastive mining could be a valid way to generate new data that would be negative for training

[JasonLovesDoggo]

Currently working on markov chains in https://github.com/jasonlovesdoggo/GopherText.

Looks promising

[circa10a]

I'd like to have this solved for the tarpit responder implementation. So I've been thinking through options:

• Markov chains
  • If we went with markov chains, we'd need some data to train the model on boot unless we had a trained model already in the GopherText library. Most linux systems have word lists in /usr/share/dict/words but that introduces a new runtime dependency and makes the application much less portable.
• Movie script cache
  • With markov chains, we would need to train a model and we could use some movie script site and train the model on application start, but we might as well just use the script and chunk them as necessary without using a markov chain at all. We could download and cache them locally on disk.
• Scrape some BS site
  • We could just download words from something like the onion and scramble the words, then cache it locally on disk for reuse.
• Embed a large word list
  • we could just package a large word list into the app using the //go:embed directive and have some functions to returns pieces of the word list data
• Random data
  • We could read random data from the /dev/urandom pipe and try to construct some words from it.

I'm a bit partial to the movie script cache personally. I think it would be funny to fetch scripts from some garbage movies, scramble it, and send that back to the scraper/attacker.

As a consumer, I wouldn't want to provide my own data. I would want something that just works out of the box which is my reasoning for the options above.

Curious what your thoughts are?

[JasonLovesDoggo]

If we went with markov chains, we'd need some data to train the model on boot unless we had a trained model already in the GopherText library. Most linux systems have word lists in /usr/share/dict/words but that introduces a new runtime dependency and makes the application much less portable.

It also wouldn't give us good results. Markov chains work off of patterns, not word lists.

there are a couple of good data sets but anything scientific would give us better results as AI content filtering tools would be less likely to ignore them. (e.g. the PMC Article Datasets)

The Chinese Learner English Corpus (CLEC) would give some interesting results.
I have some datasets on my laptop that I just had time to play with. The initial dataset (just a project Gutenberg book) is currently in https://github.com/jasonlovesdoggo/GopherText if you wish to take a look.

One bit of note is you have to think about what companies want their AIs to train on and therefore are less likely to filter out.

As for the movie idea, We could have a dataset of movie scripts, feed it into a Markov model, and use its output. The end user can have their freedom when it comes to their preference on how they want to troll an AI. (e.g. we provide a couple of "baked in pretrained markov models" but they can train they own as well using https://github.com/jasonlovesdoggo/GopherText

[JasonLovesDoggo]

see https://en.wikipedia.org/wiki/Dissociated_press

Perhaps we could even base the content upon the original non-"defended" content which we cache so the page is only ever fetched once to save CPU cycles. @circa10a thoughts?

[circa10a]

I really like the examples I saw on that Wikipedia page, but am not sure how to implement off the top.

I'm also fond of the cache idea, but would want to limit the size or make it configurable.

Neat idea!

[JasonLovesDoggo]

We could perhaps also use DI to have a pluggable markov backend. E.g. you can pick to have a smarter backend (e.g.) https://arxiv.org/pdf/1905.10649 at the cost of decreased rps or a simpler yet faster one (like the standard markov impl)