Create feeds to import content

Feeds are a pluggable system for importing or aggregating content into Drupal. Out of the box, it supports

import or aggregation from RSS feeds, Atom feeds, OPML files or CSV files
import from XML, HTML, JSON, QueryPath using Feeds extensible parsers (will be explained below)
generation of users, nodes or terms
granular mapping (for example map the "author" column of a CSV file to a text field or map the title of an RSS feed item to a term name)
multiple simultaneous configurations organized in "Feed types"
overridable default configuration for the most common use cases
aggregation (periodic import) on cron

How to create a feed type

On the Feed types overview page (admin/structure/feeds), you can see all of the Feed types that have been created so far. If this is a new install, there won't be any. To create a new Feed type click on "Structure" -> "Feed Types" and select "Add feed type." This brings you to the Feed type creation page where you have four configuration options:

Basic settings (name, description, import period)
Fetcher (from where the content is coming from)
Parser (the feed format, e.g., RSS/Atom, CSV, OPML, XML)
Processor (what type of content the feed is creating, e.g., nodes, users, taxonomy terms)

These options are described in more details below.

Mapping Sources to Targets

After you click save, you'll notice several new primary tabs at the top of the Feeds type page you just created. These include:

Mapping
- Choose how to map data from the selected feed into the content created by your import.
Manage fields, Manage form display, and Manage display
- These fields mimic the functionality and purpose they have on other content in Drupal 8. They can be edited to change the form and display of the feed, but they are not necessary for the import functionality.

Mapping

You'll need to edit the mapping of your Feed type before a Feed of that type will import content successfully. The parser you selected when creating the Feed type will affect the way the mapping will work. If you selected CSV, you'll define the "sources" based on the headers in your CSV file and map them to the target fields of your content type (or user or term). If you selected RSS/Atom, OPML, or XML Sitemap, you'll map predefined "sources" to the target fields

Creating a Feed

After you've mapped the fields you want (you can return and edit the Feed type and mapping later if you need to change anything), you'll need to create a Feed (admin/content/feed). Similarly to the Feeds type page, on this Feed overview page, you'll see all of the Feeds you've already created. Click "Add feed" to create a new Feed. If you've created multiple Feed types, you'll be asked to select which Feed type to use. If you've only created one Feed type, you'll be taken directly to that Feed creation page.

On the Feed creation page, you'll need to enter a title for the Feed and either a URL (if the Feed type has a Download fetcher), a directory path/server file (Directory fetcher), or upload a file (Upload fetcher). If using a CSV Feed type, your selections from the default secondary parser settings (delimiter and headers) will also display so that you can override them. You can also check a box to deactivate the Feed, which will prevent a recurring importer from running (it will still run if you do it manually). When activating a feed you previously deactivated, you'll also need to manually run the import in order to reset the import schedule. If you added any other custom fields on the "Manage fields" page (not the "Mapping" page), they'll show up here as well.

After you've added those fields, you can select "Save and import" or just "Save." Either way, you'll be brought to the page for the Feed you just created, where you can manually run the import and/or delete items created with that Feed.

Creating a JSON Feed

In case you want to create a non-standard feed, you'll need to install a module called Feeds extensible parsers.

Installation

Visit the module page and look at all dependencies listed there https://www.drupal.org/project/feeds_ex
Install the dependencies without composer suitable for the latest version of "Feeds extensible parsers". More info here.
Install the module "Feeds extensible parsers"

Create the feed type

Create the feed type as mentioned above and select JSONPath as the feed parser. Adjust the mappings according to the JSON format. Have a look at the following JSONPath Example:

Starting with the JSON document:

{
 "store": {
    "book": [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Tobi's story about not having humour",
        "price": 8.95
      },
      { 
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Patrick's humour-free stories",
        "price": 12.99
      },
      { 
        "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      },
      { 
        "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  }
}

Assuming you have JSONPath expressions mapped to the Title, and Author fields, your expressions would look like:

Context: $.store.book.*
    Title: title
    Author: author

Multi-lingual feeds

There is no easy solution for multi-lingual feeds as of right now. If you feel the need to create one, please contact web-team@cern.ch.

CERN Accelerating science