Hugging Face Datasets

Our official Hugging Face connector lets you import, explore and curate any dataset in Hugging Face with only a few clicks. This makes it easy for anyone to see what's in these datasets, to create embeddings from them, and to search and organize these massive and important datasets in new ways.

Importing Hugging Face Datasets into Atlas

When creating a new dataset in Atlas, you can choose Connectors as your upload option.

This will show you the list of available data connectors that integrate directly with the Atlas data upload pipeline.

Here's a video showing how to use the integration to connect directly with Hugging Face to upload Atlas from one of our example datasets.

First, select the dataset you want to import, either by clicking on one of our recommended example datasets, or searching for any dataset on Hugging Face.

We load the Datasets Viewer data preview from Hugging Face directly on the Atlas upload page so you can preview the data before uploading:

Next, a field needs to be chosen from the dataset to embed: this is the column from the dataset that will determine how the data gets arranged into semantically related clusters in the Atlas data map. We automatically select the best field for embedding from the dataset you choose, but you can choose a different field.

Then, give the dataset a name & optional description.

Finally, click Create Dataset - and you're done! The data will be ingested into Atlas and you'll get an email when your data map is ready!