There are no open source AI models, at least not useful ones (yet [1]). Open weight is not the same as open source. "Open weight" models are still just inscrutable binary blobs that you can (theoretically) run on your own computer instead of through a SAAS web app. The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data.
This is not open source and we should stop conflating the two things.
It's not just copyrighted training data. Truly open source e2e model training would include scrapers, data cleaning, all pretraining scripts, posttraining scripts, exact hardware info, etc. Open weights labs will release a sanitized version to make themselves look good / not give too much away.
I understand your quibble with terminology, but i think the “inscrutable binary blob” thing is a bit off base. You can create finetunes and post train models using only their open weights. You can’t create derivative works like that from an inscrutable binary blob
Completely agree with this, I'm sick of people conflating the two. Open-weight models should be treated no more favourably than proprietary freeware.
Sure you can run tests and benchmarks on open-weight models, but that is the extent - there is no scrutiny, no auditing for bias or copyright contamination - just a black box that you rely on for "intelligence". I'm still shocked the way people can hand over not just huge swathes of data, but also decisions of all shapes and sizes - to AI companies with no way of being able to assess how the sausage is made.
Many of the Nemotron datasets are gated behind approval, and a license agreement.
The preamble on these datasets is: "This repository is publicly accessible, but you have to accept the conditions to access its files and contents."
I don't know what others have experienced, but I requested access to multiple Nemotron datasets and those requests were ignored for months before all but one request was rejected. There's no explanation for why, nor anything I can see which would lead to a rejection. So it's purely anecdotal and YMMV, but I don't see these as being particularly open.
There are no open source AI models, at least not useful ones (yet [1]). Open weight is not the same as open source. "Open weight" models are still just inscrutable binary blobs that you can (theoretically) run on your own computer instead of through a SAAS web app. The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data.
This is not open source and we should stop conflating the two things.
[1] https://allenai.org/
because the training data is full of copyrighted works.
the answer to "what went into the training data" is "everything we could get our hands on".
I understand your quibble with terminology, but i think the “inscrutable binary blob” thing is a bit off base. You can create finetunes and post train models using only their open weights. You can’t create derivative works like that from an inscrutable binary blob
Sure you can run tests and benchmarks on open-weight models, but that is the extent - there is no scrutiny, no auditing for bias or copyright contamination - just a black box that you rely on for "intelligence". I'm still shocked the way people can hand over not just huge swathes of data, but also decisions of all shapes and sizes - to AI companies with no way of being able to assess how the sausage is made.
The preamble on these datasets is: "This repository is publicly accessible, but you have to accept the conditions to access its files and contents."
I don't know what others have experienced, but I requested access to multiple Nemotron datasets and those requests were ignored for months before all but one request was rejected. There's no explanation for why, nor anything I can see which would lead to a rejection. So it's purely anecdotal and YMMV, but I don't see these as being particularly open.