go-syndication

Report a Bug · Request a Feature . Ask a Question

Table of Contents

• About
  • Built With
• Getting Started
  • Installation
• Usage
  • Encoding and Decoding
  • Validation
  • Generic Feed/Item Types
  • Command Line Interface (CLI)
• Design
  • OpenAPI for Models
  • Validation Built In
  • Dynamic Namespace Support
  • Custom Marshal/Unmarshal
• Development
  • Setup
• Roadmap
• Support
• Project Assistance
• Contributing
• Authors & Contributors
• Security
• License

---

About

go-syndication is Go package for dealing with various feed syndication formats. It supports:

• RSS (1.x and 2.x)
• Atom
• JSONFeed
• OPML
• Various RSS/Atom extensions such as media, Dublin Core, iTunes, and GooglePlay, with more to come…

The package can read and write all formats. It includes built-in validation of elements.

Built With

• openapi-codegen.
• validator.
• bluemonday.

Getting Started

Installation

go get github.com/immanent-tech/go-syndication

Usage

Encoding and Decoding

You can decode a io.Reader containing feed data into a format using func Decode[T any](namespace string, rd io.Reader) (T, error). T would be one of *atom.Feed, *rss.RSS or *jsonfeed.Feed:

// Decode RSS feed data.
// use bytes.NewReader(data) for a []byte
rss, err := Decode[*rss.RSS]("", data)

Likewise, func Encode[T any](feed T) ([]byte, error) can be used to encode feed data:

data, err := Encode[*rss.RSS](rss)

Validation

By default, encoding/decoding performs no validation. As long as the XML data is well-formed and can be read by the Go XML parser, your feed data should be encoded/decoded. This means the package will handle invalid feed data (feeds that don't adhere to the RSS/Atom specs).

If you want to check that the feed data is valid, you can use the validation sub-package:

// Decode atom feed data.
// use bytes.NewReader(data) for a []byte
rss, err := Decode[*rss.RSS]("", data)

// Validate the atom feed.
err := validation.ValidateStruct(rss)

Generic Feed/Item Types

In addition to providing the source-specific atom.Feed, rss.RSS and jsonfeed.Feed types and their item counterparts, this library provides generic feeds.Feed and feeds.Item types, that wrap the source types with common methods for accessing their fields.

Use func NewDecoder[T any](data io.Reader) (*Feed, error) to read data into the generic object:

// data is a []byte containing an atom feed.
feed, err = feeds.NewDecoder[*rss.RSS](bytes.NewReader(data))

Feed exposes the original data as the FeedSource, which can be converted back to the original source format with a type conversion:

atom, ok := feed.FeedSource.(*atom.Feed)

This gives you the best of both worlds; a generic container with common methods for canonical fields across all formats, with access to the original source to manipulate the format directly as needed.

Command Line Interface (CLI)

A basic CLI can be found in cmd/ that can be used for basic reading/writing of feeds using the library.

To fetch and display feed data from a URL:

go run github.com/immanent-tech/go-syndication/cmd@latest fetch http://my.site/feed

To read a file containing feed data:

go run github.com/immanent-tech/go-syndication/cmd@latest parse /path/to/my/feed.xml

To lint a feed:

go run github.com/immanent-tech/go-syndication/cmd@latest lint --file=/path/to/my/feed.xml
# or --url=https://some.site/feed
# optional, add --json to get the output in JSON format

Linting by default will show validation results as well as whether the feed implements various recommended features to maximum compatibility and user experience.

All commands will auto-detect a supported feed format.

Design

OpenAPI for Models

go-syndication uses OpenAPI schemas (through oapi-codegen) to define the custom types for each syndication format and all extensions. This provides a way to have consistent, reusable types across the package.

Validation Built In

go-syndication attempts to build validation into all types using go-playground/validator. Wherever possible, types will be annotated with struct tags that then allow the validation to work.

The library aims to pass all the must test cases for Atom/RSS from feedvalidator, as well as select tests for supported extensions. You can view test results with the standard go test:

go test -v ./...

Dynamic Namespace Support

The formats in go-syndication provide dynamic namespace support. This means you can use extensions not defined in this package on top of it and get correct marshaling/unmarshaling behavior.

Custom Marshal/Unmarshal

go-syndication provides a custom Encode and Decode methods for marshaling/unmarshaling of formats (see Usage). While you can directly marshal/unmarshal, you'll lose some features (like dynamic namespaces). It's therefore recommended to always use the Encode/Decode methods in this package.

Development

Setup

1. Clone the go-syndication repo.
2. Run ./setup-feedvalidator-submodule.sh to correctly clone and filter the feedvalidator submobule to testcases directory only.

Roadmap

See the open issues for a list of proposed features (and known issues).

• Top Feature Requests (Add your votes using the 👍 reaction)
• Top Bugs (Add your votes using the 👍 reaction)
• Newest Bugs

Support

Reach out to the maintainer at one of the following places:

• GitHub issues
• Contact options listed on this GitHub profile

Project Assistance

If you want to say thank you or/and support active development of go-syndication:

• Add a GitHub Star to the project.
• Tweet about the go-syndication.
• Write interesting articles about the project on Dev.to, Medium or your personal blog.

Together, we can make go-syndication better!

Contributing

First off, thanks for taking the time to contribute! Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.

Please read our contribution guidelines, and thank you for being involved!

Authors & Contributors

The original setup of this repository is by joshuar.

For a full list of all authors and contributors, see the contributors page.

Security

go-syndication follows good practices of security, but 100% security cannot be assured. go-syndication is provided "as is" without any warranty. Use at your own risk.

For more information and to report security issues, please refer to our security documentation.

License

This project is licensed under the MIT license.

See LICENSE for more information.