gh-starred-to-opml/README.md

# gh-starred-to-opml

Generates OPML 2.0 files to track the syndication of Releases from your favorite Github repositories. In their whole or by listings.

----


## Introduction

Each repository's Releases page on Github has a [link](https://github.com/user/repository/releases.atom) to an atom file.

This **syndication feeds** allows, among other things :
  - to monitor the appearance of new releases,
  - without having to bother with notifications,
  - to monitor the disappearance of a repository.

Moreover with the recent feature of **[starred repositories lists](https://github.blog/changelog/2021-12-09-lists-are-now-available-as-a-public-beta/)**, you can choose to **follow only some personal categories** of releases.

- The **1st issue** lies in the fact that **Github's interface does not provide** any feature to **export** these atom URLs in bulk.

- However, Github's API does provide an [endpoint](https://docs.github.com/en/rest/activity/starring?apiVersion=2022-11-28#list-repositories-starred-by-a-user) to fetch all the repositories starred by an user, it's easy to generate the atom list from here (60 query per hour without auth).

- But, a **2nd issue** is that the **[API does not yet provide](https://github.com/community/community/discussions/8293)** an endpoint to **filter the starred repositories by the user's personal Lists**.

**gh-starred-to-opml is a solution**, it's a bash script that will fetch these informations for you and generate a file that can be imported in most Feed Reader applications, like the selfhosted and open source [Miniflux](https://github.com/miniflux/v2).


## Requirements

- Run with bash 5.+
- curl (sudo apt install curl)
- [jq](https://github.com/stedolan/jq/) (sudo apt install jq)
- [pup](https://github.com/ericchiang/pup) (go install github.com/ericchiang/pup@latest)


## Usage

```bash
./gh-starred-to-opml.sh -h
Generate an OPML 2.0 file to follow releases of starred repositories on Github
Default to all starred repos of the user, or a specific list with [-l list].

Syntax: ./gh-starred-to-opml.sh [-h] -u user [-l listname] [-d date] [-o filename] [-n filename]
options:
-h              Print this Help
-u string       required: Github username
-l string       optional: Github Stars List (url shortname)
-d string       optional: ISO8601 date to filter out starred before (YYYY-mm-ddTHH:MM:SSZ)
-o string.opml  optional: destination opml filename
-n string.opml  Use a previous file to generate updates,
                it does not require any other option,
                but it uses starred_at dates, not starred in a list.
```

The resulting file can be imported in a Feed Reader.
  - The first `<outline text="">` will provide the category's name.
  - `<dateCreated>` & `<dateModified>` respresent the oldest & most recent dates where repositories in the list where starred. It can help for updates.
  - The whole `<head>` is parsed by the update option (-n) to do without any other option. 

## Example

Cropped OPML extract from my Favorites list:

```xml
<?xml version="1.0" encoding="UTF-8"?>
    <opml version="2.0">
        <head>
            <title>gh_starred_tkapias_favorites_202302181136.opml</title>
            <dateCreated>Fri, 06 Sep 2019 19:35:09 GMT</dateCreated>
            <dateModified>Sun, 12 Feb 2023 11:36:12 GMT</dateModified>
            <ownerName>tkapias</ownerName>
        </head>
        <body>
            <outline text="Github - favorites - tkapias">
                <outline title="01mf02/jaq" text="01mf02/jaq" type="rss" version="ATOM1" description="A jq clone focussed on correctness, speed, and simplicity" xmlUrl="https://github.com/01mf02/jaq/releases.atom" htmlUrl="https://github.com/01mf02/jaq"></outline>
                <outline title="Airblader/xedgewarp" text="Airblader/xedgewarp" type="rss" version="ATOM1" description="Effortlessly move your cursor across monitors of different resolutions and alignments" xmlUrl="https://github.com/Airblader/xedgewarp/releases.atom" htmlUrl="https://github.com/Airblader/xedgewarp"></outline>
                <outline title="AlexandrePTJ/kemai" text="AlexandrePTJ/kemai" type="rss" version="ATOM1" description="Kimai desktop client" xmlUrl="https://github.com/AlexandrePTJ/kemai/releases.atom" htmlUrl="https://github.com/AlexandrePTJ/kemai"></outline>
                <outline title="Alexey-T/CudaText" text="Alexey-T/CudaText" type="rss" version="ATOM1" description="Cross-platform text editor, written in Lazarus" xmlUrl="https://github.com/Alexey-T/CudaText/releases.atom" htmlUrl="https://github.com/Alexey-T/CudaText"></outline>
                ...
                <outline title="zabbix/zabbix" text="zabbix/zabbix" type="rss" version="ATOM1" description="Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud." xmlUrl="https://github.com/zabbix/zabbix/releases.atom" htmlUrl="https://github.com/zabbix/zabbix"></outline>
                <outline title="zammad/zammad" text="zammad/zammad" type="rss" version="ATOM1" description="Zammad is a web based open source helpdesk/customer support system" xmlUrl="https://github.com/zammad/zammad/releases.atom" htmlUrl="https://github.com/zammad/zammad"></outline>
            </outline>
        </body>
    </opml>
```


## To be done

- Replace jq by [jaq](https://github.com/01mf02/jaq), which is much faster.
- The update option should compare the previous OPML file to find repositories that were already starred but appeared in another List. Currently the update option consider only the date where the repository was starred globaly.
Initial commit 2023-02-18 12:25:03 +01:00			`# gh-starred-to-opml`

readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			`Generates OPML 2.0 files to track the syndication of Releases from your favorite Github repositories. In their whole or by listings.`

			`----`


			`## Introduction`

readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:55:09 +01:00			`Each repository's Releases page on Github has a [link](https://github.com/user/repository/releases.atom) to an atom file.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
			`This syndication feeds allows, among other things :`
			`- to monitor the appearance of new releases,`
			`- without having to bother with notifications,`
			`- to monitor the disappearance of a repository.`

			`Moreover with the recent feature of [starred repositories lists](https://github.blog/changelog/2021-12-09-lists-are-now-available-as-a-public-beta/), you can choose to follow only some personal categories of releases.`

update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- The 1st issue lies in the fact that Github's interface does not provide any feature to export these atom URLs in bulk.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- However, Github's API does provide an [endpoint](https://docs.github.com/en/rest/activity/starring?apiVersion=2022-11-28#list-repositories-starred-by-a-user) to fetch all the repositories starred by an user, it's easy to generate the atom list from here (60 query per hour without auth).`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`- But, a 2nd issue is that the [API does not yet provide](https://github.com/community/community/discussions/8293) an endpoint to filter the starred repositories by the user's personal Lists.`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			`gh-starred-to-opml is a solution, it's a bash script that will fetch these informations for you and generate a file that can be imported in most Feed Reader applications, like the selfhosted and open source [Miniflux](https://github.com/miniflux/v2).`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00

			`## Requirements`

			`- Run with bash 5.+`
			`- curl (sudo apt install curl)`
			`- [jq](https://github.com/stedolan/jq/) (sudo apt install jq)`
			`- [pup](https://github.com/ericchiang/pup) (go install github.com/ericchiang/pup@latest)`


			`## Usage`

			```bash
			`./gh-starred-to-opml.sh -h`
			`Generate an OPML 2.0 file to follow releases of starred repositories on Github`
			`Default to all starred repos of the user, or a specific list with [-l list].`

			`Syntax: ./gh-starred-to-opml.sh [-h] -u user [-l listname] [-d date] [-o filename] [-n filename]`
			`options:`
			`-h Print this Help`
			`-u string required: Github username`
			`-l string optional: Github Stars List (url shortname)`
			`-d string optional: ISO8601 date to filter out starred before (YYYY-mm-ddTHH:MM:SSZ)`
			`-o string.opml optional: destination opml filename`
			`-n string.opml Use a previous file to generate updates,`
			`it does not require any other option,`
			`but it uses starred_at dates, not starred in a list.`
			```

			`The resulting file can be imported in a Feed Reader.`
			- The first `<outline text="">` will provide the category's name.
			- `<dateCreated>` & `<dateModified>` respresent the oldest & most recent dates where repositories in the list where starred. It can help for updates.
update readme Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 15:44:04 +01:00			- The whole `<head>` is parsed by the update option (-n) to do without any other option.
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00
			`## Example`

			`Cropped OPML extract from my Favorites list:`

			```xml
			`<?xml version="1.0" encoding="UTF-8"?>`
			`<opml version="2.0">`
			`<head>`
update example Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:34:58 +01:00			`<title>gh_starred_tkapias_favorites_202302181136.opml</title>`
readme update Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 14:21:24 +01:00			`<dateCreated>Fri, 06 Sep 2019 19:35:09 GMT</dateCreated>`
			`<dateModified>Sun, 12 Feb 2023 11:36:12 GMT</dateModified>`
			`<ownerName>tkapias</ownerName>`
			`</head>`
			`<body>`
			`<outline text="Github - favorites - tkapias">`
			`<outline title="01mf02/jaq" text="01mf02/jaq" type="rss" version="ATOM1" description="A jq clone focussed on correctness, speed, and simplicity" xmlUrl="https://github.com/01mf02/jaq/releases.atom" htmlUrl="https://github.com/01mf02/jaq"></outline>`
			`<outline title="Airblader/xedgewarp" text="Airblader/xedgewarp" type="rss" version="ATOM1" description="Effortlessly move your cursor across monitors of different resolutions and alignments" xmlUrl="https://github.com/Airblader/xedgewarp/releases.atom" htmlUrl="https://github.com/Airblader/xedgewarp"></outline>`
			`<outline title="AlexandrePTJ/kemai" text="AlexandrePTJ/kemai" type="rss" version="ATOM1" description="Kimai desktop client" xmlUrl="https://github.com/AlexandrePTJ/kemai/releases.atom" htmlUrl="https://github.com/AlexandrePTJ/kemai"></outline>`
			`<outline title="Alexey-T/CudaText" text="Alexey-T/CudaText" type="rss" version="ATOM1" description="Cross-platform text editor, written in Lazarus" xmlUrl="https://github.com/Alexey-T/CudaText/releases.atom" htmlUrl="https://github.com/Alexey-T/CudaText"></outline>`
			`...`
			`<outline title="zabbix/zabbix" text="zabbix/zabbix" type="rss" version="ATOM1" description="Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud." xmlUrl="https://github.com/zabbix/zabbix/releases.atom" htmlUrl="https://github.com/zabbix/zabbix"></outline>`
			`<outline title="zammad/zammad" text="zammad/zammad" type="rss" version="ATOM1" description="Zammad is a web based open source helpdesk/customer support system" xmlUrl="https://github.com/zammad/zammad/releases.atom" htmlUrl="https://github.com/zammad/zammad"></outline>`
			`</outline>`
			`</body>`
			`</opml>`
			```


			`## To be done`

readme update: todo Signed-off-by: Tomasz Kapias <tkapias@git.tkapias.net> 2023-02-18 21:24:28 +01:00			`- Replace jq by [jaq](https://github.com/01mf02/jaq), which is much faster.`
			`- The update option should compare the previous OPML file to find repositories that were already starred but appeared in another List. Currently the update option consider only the date where the repository was starred globaly.`