Twitter crossposter

timconsidine

I understand this might be of interest only to a niche audience.
But here goes.

I hate almost everything about Twitter, particularly being monetised by them, spammed by many others and the "twitter bubbles" that get created. But ... I am "required" to have a presence there on one project and taking flak for not doing so.
I read tweets from accounts which I have to follow via RSS feed from Nitter (self-hosted) so I don't get contaminated.

These seem to be the primary apps for posting content to Twitter from "outside".
https://github.com/renatolond/mastodon-twitter-poster
https://gitlab.com/fedstoa/moa

But I can't get them going on a non-Cloudron VPS.
Wondering if they might appeal to a Cloudron audience, even if only a sub-set.

Of course, I could use the public instances of the above but (a) I believe in self-hosting, (b) don't want to load other peoples' servers, (c) need to support a number of accounts.

Alternatively, do let me know if you know of other approaches to solving this problem.

robi

Why not set up an N8N workflow to take something from say RSS and post it to Twitter?

timconsidine

@robi thanks.
N8N didn't occur to me.
Don't really know it but I can take a look.

robi

@timconsidine You are very welcome!

I will be interested in some of the things you learn there as well, as I'd like to connect Insta posting to Twitter posting and not necessarily from the same account(s).

timconsidine

@robi good suggestion about n8n
I got a basic flow working.
I used Zapier a lot before, so should understand n8n but struggled a bit to understand its components initially. But persevered. n8n looks more flexible/powerful than Zapier.

I set up a flow with :

interval node as trigger every 15 mins
RSS Read node : I didn't know that Mastodon make it so easy to get a feed of a user's posts with just https://<instance.name>/users/<username>
SET to create a new JSON entry e.g. tweetString to get the rss feed elements
FUNCTION to strip stray html from tweetString
TWITTER node to submit the tweetString : I found n8n UI a bit unclear : you have to set up the oAuth in apps.twitter.com before setting up the n8n node (I read it as after)

Still have to deal with adding images and links from the rss feed to the tweetString, but I think it is doable. Certainly works well for plain text toot-->tweet.

robi

@timconsidine

Excellent progress! You can find more inspiration from the N8N wokflows community https://n8n.io/workflows/

Their community forum also had this that will be useful.
https://community.n8n.io/t/twitter-error-response-403-tweet-needs-to-be-a-bit-shorter/3207/5

jdaviescoates

@timconsidine said in Twitter crossposter:

These seem to be the primary apps for posting content to Twitter from "outside".
https://github.com/renatolond/mastodon-twitter-poster

Yeah, I use the hosted version of that here:
https://crossposter.masto.donte.com.br/

But it'd be cool to be able to self-host it too.

Although sounds like you're getting there with just doing it with N8N too, which is great!

But this crossposted app does still have some nice features/ settings (e.g. whether or not to crosspost RTs etc) which I'd guess might be harder to replicate using N8N.

robi

@jdaviescoates said in Twitter crossposter:

But this crossposted app does still have some nice features/ settings (e.g. whether or not to crosspost RTs etc) which I'd guess might be harder to replicate using N8N.

That's just a condition (IF) block to filter out RTs, etc. Easy Peazy

timconsidine

@jdaviescoates I may have spoken too soon.
n8 is working but Twitter Oauth is playing up.
Strange as it was working last night.

jdaviescoates

@timconsidine perhaps you've hit some Twitter throttling or something? I'm pretty sure there are limits to how many calls you can do.

timconsidine

@jdaviescoates yes indeed there are.
But normally they give you a specific error message.
At least they did when I played with some PHP scripts.
The error coming back is that OAuth is not connected.
Even thought it was.
Even though I deleted and created new credentials and new connection.
I suspect a network error or a domain block.
Investigating.

timconsidine

@jdaviescoates said in Twitter crossposter:

Yeah, I use the hosted version of that here:
https://crossposter.masto.donte.com.br/

Yes this is cool, and I have used it before on a test basis.
I'm just trying to do same on self-hosted basis.
One of the links in original post is the source to that site
https://github.com/renatolond/mastodon-twitter-poster

I'll have another bash at installing it this weekend, because as you say, it has some neat features.

timconsidine

@timconsidine I'm an idiot.
Or my browser is.
Blocks pop-ups without showing any indication of blocked pop-up. Twitter pop-up to authorise n8n app was not shown.
Switching browser showed it.
What a dumb schmuck I am !

timconsidine

@timconsidine the ongoing saga :
despite my idiocy earlier (cough, let's draw a veil over that), it seems there is a bug in n8n v0.139

It seems to be a bug introduced in the latest version of n8n 0.139.1. Please downgrade to 0.138 or set the env variable N8N_USE_DEPRECATED_REQUEST_LIB=true

My Twitter node authenticates fine and posts static text, but fails to post the result of a workflow. Kinda useless in its current state. But there is a workaround, praise be to someone on high.

Just FYI in case any others are struggling as I have been.
I guess better post this and future in n9n forum, rather than here.

timconsidine

@robi said in Twitter crossposter:

Why not set up an N8N workflow to take something from say RSS and post it to Twitter?

This has worked well, so thanks again.
Screenshot 2021-09-26 at 12.32.38.png

Maybe not as full-featured as moa or renatolond
But doing what I need and I think easier to maintain in future.

Need to add x-post a mastodon image and to add a schedule for repeating running every x hours.

robi

@timconsidine So this is after the config tweak?

Yes, images/videos would be a useful/required thing.

Why are there two twitter accounts?

timconsidine

@robi yes, I set the environment variable rather than downgrade.
And then it just sailed through.

The 1st twitter account is more of an organisation account and the 2nd twitter account is an individual account.
I mostly wanted to test that I could auto-RT another account.
The Set1 grabs the tweet id returned by Twitter on posting, and the 2nd account RT's that tweet id.
Not hugely useful but I wanted to prove it could be done.
The Wait is just to provide an 'air gap' of sorts in case it all goes through too quickly for systems to cope.

Screenshot 2021-09-26 at 23.08.15.png

Images are now added.
Docs/forum say HTTP Request node needed for that.

And as the RSS feed returns a list of toots, I needed to avoid duplicate posts. So I added an IF node which tests if a toot is older than xxx milliseconds (5 mins in my case), and only continues if it is newer. If none newer, the flow expires (no false tree set up).

Now I just need to add a cron or scheduler node.
But leaving it manual firing for now to test edge cases which different content might show errors on.

timconsidine

@robi said in Twitter crossposter:

I'd like to connect Insta posting to Twitter posting and not necessarily from the same account(s)

I don't use Instagram but ironically just found one of my 'persons of interest' does, and mostly tweets links to Insta posts. So you have to follow them to see content. So I may look into this.

Brief research shows you can use Bibliogram to get an RSS feed of Instagram posts from an Insta account. Based on that, it seems at first glance that it is possible to use similar workflow to my Mastodon2Twitter to achieve that.

I tried briefly to self-host Bibliogram but installation failed, so as Instagram is low usage for me, I'm just using a hosted Bibliogram (https://bibliogram.snopyta.org/) to give me the RSS feed.

EDIT : just seen Bibliogram is a supported app on Yunohost. I don't have a Yunohost server .... yet !

robi

@timconsidine Beautiful, nice find on bibliogram.

I was looking at other such services and Nitter showed up, with a bit more understanding of why it exists as described here:
https://nitter.1d4.us/about

This also means that if you replace your Twitter node with a Nitter node, you'll likely have a better experience posting to Twitter. (no rate limits of dev account required)

timconsidine

@robi said in Twitter crossposter:

if you replace your Twitter node with a Nitter node

oooo, didn't know I could do that !

I also noticed RSS-Bridge supports an Instagram feed, but haven't played with it yet.

robi

@timconsidine I may be wrong as so far it seems as a read-only config, not sure if one can post to Twitter as well.

timconsidine

@robi I don't think you can post via Nitter, it is has no login, just read-only.
But your post made me think because as you say, the dev accounts have rate limits.
Limits are not so bad, but they're there, and for some use cases, could be a problem, e.g. analysis and "tweet intelligence".
My current n8n is great for posting, and I won't hit rate limits.
But for other use, a lot of tweet fetching might.
That's where Nitter can come into play.

robi

@timconsidine https://github.com/FriendsOfREDAXO/feeds might be good intermediary step so you get nice RSS feeds for N8N.

timconsidine

@robi

rmdes

@timconsidine Hey, I'm trying to achieve a similar scenario but I'm not getting far, would you by any chance share your n8n workflow ?

timconsidine

@rmdes sure, let me copy and upload here in a while

timconsidine

@rmdes here is my n8n workflow for taking an RSS feed from a mastodon user and posting that to twitter

It can sure be made more sophisticated but it works in basic form

Main point is that it requires a Twitter OAuth login, which is usually (always?) obtained by having a Twitter API registration which gives you the credentials needed to be input in your n8n.
It is not your normal Twitter login.

Getting a Twitter API account is not difficult but it's not entirely straight forward. They seem to randomly accept/reject for unclear reasons. But if you have one already or are patient to go through the process, you should be good to go.

Cam't seem to attach a file so you may have to copy/paste this into a .json file and then import that into your new n8n workflow.

{
  "name": "mastodon2twitter",
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        -10,
        30
      ]
    },
    {
      "parameters": {
        "url": "https://mastodon.domain/users/username"
      },
      "name": "RSS Feed Read",
      "type": "n8n-nodes-base.rssFeedRead",
      "typeVersion": 1,
      "position": [
        160,
        30
      ],
      "notesInFlow": false,
      "notes": "account username"
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "tweetString",
              "value": "={{$json[\"contentSnippet\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        320,
        30
      ],
      "notesInFlow": true,
      "notes": "compose tweetString"
    },
    {
      "parameters": {
        "functionCode": "// Code here will run only once, no matter how many input items there are.\n// More info and help: https://docs.n8n.io/nodes/n8n-nodes-base.function\n// Loop over inputs \n\nvar date_run = new Date();\ncnt=0;\nfor (item of items) {\n  var item_date = new Date(item.json.isoDate);\n  if (date_run - item_date > 300000) {\n   //delete items[cnt];\n   item.json.tweetString = \"expired\";\n  } \n cnt++;\n}\n\n//items.length = 1;\n\n// You can write logs to the browser console\n//console.log('Done!');\n\nreturn items;\n\n"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        490,
        30
      ],
      "notesInFlow": true,
      "notes": "set tweet expired if >5m"
    },
    {
      "parameters": {
        "conditions": {
          "string": [
            {
              "value1": "={{$json[\"tweetString\"]}}",
              "operation": "notEqual",
              "value2": "expired"
            }
          ]
        },
        "combineOperation": "any"
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        650,
        40
      ],
      "notesInFlow": true,
      "notes": "check not expired"
    },
    {
      "parameters": {
        "url": "={{$json[\"enclosure\"][\"url\"]}}",
        "responseFormat": "file",
        "options": {}
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        910,
        -50
      ]
    },
    {
      "parameters": {
        "text": "={{$node[\"Set\"].json[\"tweetString\"]}}",
        "additionalFields": {
          "attachments": "data"
        }
      },
      "name": "Twitter",
      "type": "n8n-nodes-base.twitter",
      "typeVersion": 1,
      "position": [
        1110,
        -50
      ],
      "notesInFlow": false,
      "credentials": {
        "twitterOAuth1Api": "Twitter OAuth account"
      },
      "notes": "twitter"
    },
    {
      "parameters": {
        "triggerTimes": {
          "item": [
            {
              "mode": "everyX",
              "value": 5,
              "unit": "minutes"
            }
          ]
        }
      },
      "name": "Cron",
      "type": "n8n-nodes-base.cron",
      "typeVersion": 1,
      "position": [
        -20,
        -130
      ],
      "notesInFlow": true,
      "notes": "each 5m"
    }
  ],
  "connections": {
    "RSS Feed Read": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request": {
      "main": [
        [
          {
            "node": "Twitter_2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cron": {
      "main": [
        [
          {
            "node": "RSS Feed Read",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": true,
  "settings": {
    "saveExecutionProgress": true,
    "saveManualExecutions": true
  },
  "id": 6
}

218 lines long so be sure to copy all, or send me DM to try to exchange the file

rmdes

@timconsidine hey there, thanks a lot for the recipe, going to test this out !!

can you explain why the Twitter node is not connected to anything ?
also can you explain what the 5 min function is doing ?

I'm asking because even though I have setup everything, (have my own twitter credentials ready and obviously changed the RSS source but I see no posts being created, even though the recipe clearly fetch my RSS feed and at the end of the workflow the twitter node "see" the content of the last theoretical tweet the workflow should have made...but nothing get's out on twitter..

No errors on any of the workflow, it all seems to be great, but no output ?!

timconsidine

@rmdes

The Twitter node is not connected to anything as output, because it is the end of the process.
Are you saying it is not connected on input side when you import the workflow ? But I think it is because our screenshot some incoming values.

The 5min function was was simply to filter out posts on Mastodon made >5 mins ago, so that the workflow did not attempt to post "old" posts to Twitter. My understanding is that Twitter would not detect a duplicate tweet (although it does detect duplicate RTs). So the function just takes Mastodon posts from <5 mins age. This is also related to the cron running the workflow every 5 mins.

Of course you can adjust these values or remove them, but you might get duplicate tweets.

Twitter should give an error if there is a problem, which you can see in the n8n step. Nothing showing there ?

rmdes

@timconsidine No no the input is connected just fine, maybe it's just me I had not seen a workflow like this before and it feels weird that nothing goes published..not even sure where to look since I don't have any errors, all executions are success...

rmdes

@timconsidine by the way, twitter does even more, it detect duplicates tweets also based on content, title + link, if the attempt is made to tweet an item that was already tweeted via the application, the twitter api return an "duplicate tweet" error (that's with my previous python based RSS to twitter experiment) when the duplicate tweet is detected, twitter simply reject the second attempt at tweeting the same thing twice.

I think I will have to increase the 5 min because, provided I don't have duplicates I don't mind going back 1h ago to tweet items, I was under the impression that maybe one of the reason it does not tweet the output is maybe because the time span is too short ?

rmdes

@timconsidine I increased the time to 1800.000, so 30 minutes, but I don't see any difference, wondering what's happening
edit : I have 400+ json item that each part of the workflow can fetch, see, but nothing gets out...

robi

@rmdes Can you just drag a line from your HTTP block to the twitter block?

timconsidine

@rmdes yes in that diagram there is no input to the Twitter node

Drag from right hand side of HTTP Request node to the left hand side of Twitter node

rmdes

@timconsidine When I do so I get this error on the HTTP Request node when I execute the workflow :

The "url" argument must be of type string. Received undefined

{
"status": "rejected",
"reason": {
"code": "ERR_INVALID_ARG_TYPE",
"cause": {
"code": "ERR_INVALID_ARG_TYPE"
},
"error": {
"code": "ERR_INVALID_ARG_TYPE"
},
"options": {
}
}
}

Actually I don't understand what this Node is doing, can't I send the data directly to twitter after the IF ?

timconsidine

@rmdes yes you can send to Twitter node after the IF node providing the tweet is text only. The HTTP node is used to capture any images in the Mastodon tweet and attach them (as binary data I think)

You may just need to "re-map" the HTTP node links from IF node, or re-map the links in the Twitter node to the HTTP output.

I usually step through each node from the start (using that node's run icon, not the whole workflow run icon), and in each subsequent node, check the Input parameters to select the incoming data and map to the node's parameters.

Then I do an end-to-end test.

My workflow provides the general schema, but I had problems on importing other workflows (different use cases). I just deleted each imported node one-by-one and recreated the mode manually, linking to the previous node as explained above.

rmdes

@timconsidine Thanks & sorry for late reply, I managed to get it working, but I'm not happy with the result in my use case, in fact I'm not happy with N8, the main issue being missing key features to validate their "no code" approach, I think some key elements are missing that render any attempt to use n8 for automation-publishing a real P in the A..

I tried dozens of implementations, taking on recipe from others to rebuild my own and after several months of testing, I see duplicates, n8 stuck in loop publishing over and over the same stuff, I may be doing something wrong but the moment you add complexity to the recipe, let's say : add some RSS feeds feeding the same output, the situation becomes a mess.

timconsidine

@rmdes interesting.
I did not add much complexity, so maybe didn't get to those problems.

rmdes

@timconsidine I'll try to simplify, isolate each feed in their own workflow then

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Twitter crossposter