gh-starred-to-opml/README.md

# gh-starred-to-opml

Generates OPML 2.0 files to track the syndication of Releases (binaries or tags) from your favorite Github repositories. In their whole or by listings.

----


## Introduction

Each repository's Releases page on Github is linked to 2 atom files: [releases](https://github.com/user/repository/releases.atom) and [tags](https://github.com/user/repository/tags.atom) .

Those **syndication feeds** allows, among other things :
  - to monitor the appearance of new releases,
  - without having to bother with notifications,
  - to monitor the disappearance of a repository.

Moreover with the recent feature of **[starred repositories lists](https://github.blog/changelog/2021-12-09-lists-are-now-available-as-a-public-beta/)**, you can choose to **follow only some personal categories** of releases.

- The **1st issue** lies in the fact that **Github's interface does not provide** any feature to **export** these atom URLs in bulk.

- However, Github's API does provide an [endpoint](https://docs.github.com/en/rest/activity/starring?apiVersion=2022-11-28#list-repositories-starred-by-a-user) to fetch all the repositories starred by an user, it's easy to generate the atom list from here (60 query per hour without auth).

- But, a **2nd issue** is that the **[API does not yet provide](https://github.com/community/community/discussions/8293)** an endpoint to **filter the starred repositories by the user's personal Lists**.

- In addition, **Github's API has a rate limit for anonymous requests** of 60 per hour and per IP. You need to use a Token to push this limit between 1000 and 5000 per hour and per IP.

**gh-starred-to-opml is a solution**, it's a bash script that will fetch these informations for you and generate a file that can be imported in most Feed Reader applications, like the selfhosted and open source [Miniflux](https://github.com/miniflux/v2).


## Requirements

- Run with bash 5.+
- curl (sudo apt install curl)
- [jq](https://github.com/stedolan/jq/) (sudo apt install jq)
- [pup](https://github.com/ericchiang/pup) (go install github.com/ericchiang/pup@latest)
- A Github [personal access token](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api?apiVersion=2022-11-28#basic-authentication).

## Usage

```bash
./gh-starred-to-opml.sh -h
Generate an OPML 2.0 file to follow releases of starred repositories on Github
Default to all starred repos of the user, or a specific list with [-l list].
Uses the Github API to check lists and if the starred repos contain Release (preferred) or Tag entries.

Syntax: ./gh-starred-to-opml.sh [-h] -u user [-l listname] [-d date] [-o filename] [-n filename] -t token
options:
-h              Print this Help
-u string       required: Github username
-l string       optional: Github Stars List (url shortname)
-d string       optional: ISO8601 date to filter out starred before (YYYY-mm-ddTHH:MM:SSZ)
-o string.opml  optional: destination opml filename
-n string.opml  Use a previous file to generate updates,
                it does not require any other option,
                but it uses starred_at dates, not starred in a list.
-t string       required: Github API token to avoid rate limits.
```

- The resulting file can be imported in a Feed Reader.

## Example

Cropped OPML extract from my Favorites list:

```xml
<?xml version="1.0" encoding="UTF-8"?>
    <opml version="2.0">
        <head>
            <title>gh_starred_tkapias_favorites_202302181136.opml</title>
            <dateCreated>Fri, 06 Sep 2019 19:35:09 GMT</dateCreated>
            <dateModified>Sun, 12 Feb 2023 11:36:12 GMT</dateModified>
            <ownerName>tkapias</ownerName>
        </head>
        <body>
            <outline text="Github - favorites - tkapias">
                <outline title="01mf02/jaq" text="01mf02/jaq" type="rss" version="ATOM1" description="A jq clone focussed on correctness, speed, and simplicity" xmlUrl="https://github.com/01mf02/jaq/releases.atom" htmlUrl="https://github.com/01mf02/jaq"></outline>
                <outline title="Airblader/xedgewarp" text="Airblader/xedgewarp" type="rss" version="ATOM1" description="Effortlessly move your cursor across monitors of different resolutions and alignments" xmlUrl="https://github.com/Airblader/xedgewarp/releases.atom" htmlUrl="https://github.com/Airblader/xedgewarp"></outline>
                <outline title="AlexandrePTJ/kemai" text="AlexandrePTJ/kemai" type="rss" version="ATOM1" description="Kimai desktop client" xmlUrl="https://github.com/AlexandrePTJ/kemai/releases.atom" htmlUrl="https://github.com/AlexandrePTJ/kemai"></outline>
                <outline title="Alexey-T/CudaText" text="Alexey-T/CudaText" type="rss" version="ATOM1" description="Cross-platform text editor, written in Lazarus" xmlUrl="https://github.com/Alexey-T/CudaText/releases.atom" htmlUrl="https://github.com/Alexey-T/CudaText"></outline>
                ...
                <outline title="zabbix/zabbix" text="zabbix/zabbix" type="rss" version="ATOM1" description="Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud." xmlUrl="https://github.com/zabbix/zabbix/releases.atom" htmlUrl="https://github.com/zabbix/zabbix"></outline>
                <outline title="zammad/zammad" text="zammad/zammad" type="rss" version="ATOM1" description="Zammad is a web based open source helpdesk/customer support system" xmlUrl="https://github.com/zammad/zammad/releases.atom" htmlUrl="https://github.com/zammad/zammad"></outline>
            </outline>
        </body>
    </opml>
```


## To be done

- Replace jq by [jaq](https://github.com/01mf02/jaq), which is much faster.
Initial commit 2023-02-18 12:25:03 +01:00			`# gh-starred-to-opml`

updated for Github's API Token 2024-02-07 20:16:02 +01:00			`Generates OPML 2.0 files to track the syndication of Releases (binaries or tags) from your favorite Github repositories. In their whole or by listings.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
			`----`


			`## Introduction`

updated for Github's API Token 2024-02-07 20:16:02 +01:00			`Each repository's Releases page on Github is linked to 2 atom files: [releases](https://github.com/user/repository/releases.atom) and [tags](https://github.com/user/repository/tags.atom) .`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`Those syndication feeds allows, among other things :`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			`- to monitor the appearance of new releases,`
			`- without having to bother with notifications,`
			`- to monitor the disappearance of a repository.`

			`Moreover with the recent feature of [starred repositories lists](https://github.blog/changelog/2021-12-09-lists-are-now-available-as-a-public-beta/), you can choose to follow only some personal categories of releases.`

update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- The 1st issue lies in the fact that Github's interface does not provide any feature to export these atom URLs in bulk.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- However, Github's API does provide an [endpoint](https://docs.github.com/en/rest/activity/starring?apiVersion=2022-11-28#list-repositories-starred-by-a-user) to fetch all the repositories starred by an user, it's easy to generate the atom list from here (60 query per hour without auth).`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- But, a 2nd issue is that the [API does not yet provide](https://github.com/community/community/discussions/8293) an endpoint to filter the starred repositories by the user's personal Lists.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`- In addition, Github's API has a rate limit for anonymous requests of 60 per hour and per IP. You need to use a Token to push this limit between 1000 and 5000 per hour and per IP.`

update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`gh-starred-to-opml is a solution, it's a bash script that will fetch these informations for you and generate a file that can be imported in most Feed Reader applications, like the selfhosted and open source [Miniflux](https://github.com/miniflux/v2).`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00

			`## Requirements`

			`- Run with bash 5.+`
			`- curl (sudo apt install curl)`
			`- [jq](https://github.com/stedolan/jq/) (sudo apt install jq)`
			`- [pup](https://github.com/ericchiang/pup) (go install github.com/ericchiang/pup@latest)`
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`- A Github [personal access token](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api?apiVersion=2022-11-28#basic-authentication).`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
			`## Usage`

			```bash
			`./gh-starred-to-opml.sh -h`
			`Generate an OPML 2.0 file to follow releases of starred repositories on Github`
			`Default to all starred repos of the user, or a specific list with [-l list].`
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`Uses the Github API to check lists and if the starred repos contain Release (preferred) or Tag entries.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`Syntax: ./gh-starred-to-opml.sh [-h] -u user [-l listname] [-d date] [-o filename] [-n filename] -t token`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			`options:`
			`-h Print this Help`
			`-u string required: Github username`
			`-l string optional: Github Stars List (url shortname)`
			`-d string optional: ISO8601 date to filter out starred before (YYYY-mm-ddTHH:MM:SSZ)`
			`-o string.opml optional: destination opml filename`
			`-n string.opml Use a previous file to generate updates,`
			`it does not require any other option,`
			`but it uses starred_at dates, not starred in a list.`
updated for Github's API Token 2024-02-07 20:16:02 +01:00			`-t string required: Github API token to avoid rate limits.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			```

updated for Github's API Token 2024-02-07 20:16:02 +01:00			`- The resulting file can be imported in a Feed Reader.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
			`## Example`

			`Cropped OPML extract from my Favorites list:`

			```xml
			`<?xml version="1.0" encoding="UTF-8"?>`
			`<opml version="2.0">`
			`<head>`
update example Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:34:58 +01:00			`<title>gh_starred_tkapias_favorites_202302181136.opml</title>`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			`<dateCreated>Fri, 06 Sep 2019 19:35:09 GMT</dateCreated>`
			`<dateModified>Sun, 12 Feb 2023 11:36:12 GMT</dateModified>`
			`<ownerName>tkapias</ownerName>`
			`</head>`
			`<body>`
			`<outline text="Github - favorites - tkapias">`
			`<outline title="01mf02/jaq" text="01mf02/jaq" type="rss" version="ATOM1" description="A jq clone focussed on correctness, speed, and simplicity" xmlUrl="https://github.com/01mf02/jaq/releases.atom" htmlUrl="https://github.com/01mf02/jaq"></outline>`
			`<outline title="Airblader/xedgewarp" text="Airblader/xedgewarp" type="rss" version="ATOM1" description="Effortlessly move your cursor across monitors of different resolutions and alignments" xmlUrl="https://github.com/Airblader/xedgewarp/releases.atom" htmlUrl="https://github.com/Airblader/xedgewarp"></outline>`
			`<outline title="AlexandrePTJ/kemai" text="AlexandrePTJ/kemai" type="rss" version="ATOM1" description="Kimai desktop client" xmlUrl="https://github.com/AlexandrePTJ/kemai/releases.atom" htmlUrl="https://github.com/AlexandrePTJ/kemai"></outline>`
			`<outline title="Alexey-T/CudaText" text="Alexey-T/CudaText" type="rss" version="ATOM1" description="Cross-platform text editor, written in Lazarus" xmlUrl="https://github.com/Alexey-T/CudaText/releases.atom" htmlUrl="https://github.com/Alexey-T/CudaText"></outline>`
			`...`
			`<outline title="zabbix/zabbix" text="zabbix/zabbix" type="rss" version="ATOM1" description="Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud." xmlUrl="https://github.com/zabbix/zabbix/releases.atom" htmlUrl="https://github.com/zabbix/zabbix"></outline>`
			`<outline title="zammad/zammad" text="zammad/zammad" type="rss" version="ATOM1" description="Zammad is a web based open source helpdesk/customer support system" xmlUrl="https://github.com/zammad/zammad/releases.atom" htmlUrl="https://github.com/zammad/zammad"></outline>`
			`</outline>`
			`</body>`
			`</opml>`
			```


			`## To be done`

readme update: todo Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 21:24:28 +01:00			`- Replace jq by [jaq](https://github.com/01mf02/jaq), which is much faster.`