No longer any remote follows on instance, but inexplicable content from remote accounts still appearing in Federated timeline

[FediVideos]

Steps to reproduce the problem

1. Unfollow all remote accounts on all of an instance's accounts
2. Wait a few weeks, go to Federated timeline
3. Observe unboosted remote posts still appearing in Federated
   ...

Expected behaviour

If there are no remote follows, the federated timeline should only show remote posts if they're boosted by local accounts

Actual behaviour

The federated timeline shows unboosted remote posts

Detailed description

I run a single-user instance which is mainly for posting advice and answering questions. My personal account is not on it.

To save resources I unfollowed all remote accounts, this was several weeks ago. The instance is on a managed hosting service and the hosting company verified from the database that there are zero remote follows. When I browse the Federated timeline, am expecting to only see remote posts that I've boosted, but am actually seeing unboosted posts from many remote accounts.

I do not have any relays set up.

One possible theory: Some of Federated is recent posts from accounts I used to follow a long time ago, and the other recent posts are the kind of content these ex-follows might be boosting now. It seems like for some reason my instance is still pulling in content from remote accounts I used to follow a long time ago?

Is Mastodon's federation system not unfollowing accounts properly? Or does it keep federating an account's content if it's ever been followed, even if no one on the instance is following that account?

(By the way, I discussed this first with the managed hosting company, they looked into it and they had no idea why this was happening. They encouraged me to file an issue for this, and to let them know the link so they can resolve this if needed.)

Mastodon instance

social.growyourown.services

Mastodon version

v4.2.8

Technical details

If this is happening on your own Mastodon server, please fill out those:
(I am not sure about Ruby and NodeJS versions, it's running on the managed hosting service masto.host.)

[ClearlyClaire]

Mastodon will display in the “public timeline” public posts that are known to it, not only those that were boosted. This means if a local or followed user replied to a remote post, this remote post will be fetched.

It is difficult making any theory without more data, I think it would help having examples of posts that have reached your instance with no obvious path.

[FediVideos]

I'm the only user on the instance, and I'm not following any remote accounts, so the only way remote posts could be known to my instance is through boosting and replying and URL searches?

I haven't interacted in any way with most of the posts in Federated.

For example I haven't interacted in any way with any of these, but they are in my Federated timeline:

https://indieauthors.social/@Klepsis/112391540624592317
https://mastodon.art/@4rtef8ct/112388879092462741
https://bsd.network/@claudiom/112383920901639130
https://mastodon.green/@Philsturgeon/112382300981701587
https://flipboard.social/@coffeegeek/112381239462792243
https://mastodon.social/@gutenberg_org/112376339669544981
https://mastodon.social/@thejikz/112374443298333203
https://fediversity.site/item/9cb33387-8b53-40a6-a162-be123a69f252
https://ursal.zone/@walsturz/112371814450703562
https://hachyderm.io/@molly0xfff/112367788538433782

...these are just randomly taken from the top of the latest Federated feed, there are lots more posts like this from other accounts on many different remote instances.

[hugogameiro]

@ClearlyClaire I just talked with @FediVideos and they allowed me to share any details you need to debug this situation. Feel free to ping me if you think that would be helpful. Thanks.

[ClearlyClaire]

Looking at some of these:

For example I haven't interacted in any way with any of these, but they are in my Federated timeline:

https://indieauthors.social/@Klepsis/112391540624592317
https://mastodon.art/@4rtef8ct/112388879092462741

Those posts are pinned posts, so they would be pulled whenever discovering their author account (e.g. through a post that mentions them).

https://bsd.network/@claudiom/112383920901639130

This one seems to have been pulled because of this reply: https://social.growyourown.services/@elmussol@streams.elsmussols.net/112384120317230513

https://mastodon.green/@Philsturgeon/112382300981701587

This one seems to have been pulled because of this reply: https://social.growyourown.services/@elmussol@streams.elsmussols.net/112382825912701927

https://flipboard.social/@coffeegeek/112381239462792243

Might have been pulled from https://social.growyourown.services/@billstatler@forum.statler.ws/112384554089486587

https://mastodon.social/@gutenberg_org/112376339669544981

Seems to have been pulled from https://social.growyourown.services/@elmussol@streams.elsmussols.net/112376428699670315

---

So I think at least elmussol@streams.elmussols.net is still sending you updates as if you were following him. It may also be the case of billstatler@forum.statler.ws. They both seem to use the same fediverse software, which might explain both your server accepting their posts despite you not following them, and them still sending you posts despite there being no obvious reason to.

Looking at the database confirms that posts from @elmussol are specifically sent to FediTips, presumably as if you were still following him.

[FediVideos]

Thank you so much for taking the time to check this, really appreciated 🙏

Does that mean Streams accounts could be used to spam a Masto instance? 😬

I have never followed either of these accounts, and never heard of them before. AFAIK I have never interacted with them. It sounds like my instance is accepting whatever they want to send, without any reason to do so?

Sounds like a security vulnerability if remote instances can push unrequested content like this by using custom software? (Obviously hope I'm wrong about this, just this is what it sounds like?)

[FediVideos]

Okay, it looks like the Streams software sends content to follows instead of just followers:

https://codeberg.org/streams/streams/issues/144

If a Streams user follows you, they may unknowingly send content to your instance even if you have never followed or interacted with them, even if the Streams user hasn't mentioned you in the post.

UPDATE: Looks like Streams is investigating this, followers' content isn't supposed to go to Mastodon accounts, only Streams accounts.

[FediVideos]

...and now the Streams dev has changed their mind and says they aren't going to fix it, they say it's Mastodon's fault for accepting the content:

https://codeberg.org/streams/streams/issues/144#issuecomment-1813593

Is there anything that can be done apart from blocking/defederating Streams accounts/instances?

[ClearlyClaire]

Hi! I added more information to that issue, and had Hugo add some instrumentation code so that we can investigate further on the next message from @elmussol that reaches your server. I also wrote to @elmussol to get more information from their end.

[FediVideos]

Thank you for looking into this further, and for working with Hugo on this. Let's hope some more information comes to light.

[FediVideos]

Streams have now locked their thread on this issue and accused me of not knowing who I was following:

https://codeberg.org/streams/streams/issues/144

I guess there is nothing more Mastodon can do from this end? If so, I will close the issue.

[FediVideos]

Update on this: I suspended all of the Streams instances that I could find federating with me, and immediately all of the unexplained posts stopped. Not just Streams posts but all of the unexplained posts from non-Streams accounts too. My instance's federated timeline is now totally back in my control.

So, it was 100% definitely Streams accounts following me that were the root cause of this. Streams accounts were pushing content to me from Streams, Mastodon, Friendica, GoToSocial etc that I had no connection with, but it was only the Streams instances that I had to suspend in order to stop all of it happening. (I didn't have to block any non-Streams instances, so the non-Streams instances were not doing this at all.)

As Streams developers are refusing to engage on this, I guess the only recommendation is admins who are receiving unwanted unexplained content to their instances should try seeing if it's connected to Streams instances and potentially suspend them if necessary.

I'll close this issue. Thanks again for your time @ClearlyClaire, it would have been impossible to know it was Streams causing it without your analysis of all this 🙏 and thank you also @hugogameiro for being so proactive about checking the database etc 👏

[ClearlyClaire]

Some more information on this:

• Streams developers commented that Stream's behavior was changed so this does not occur again: https://codeberg.org/streams/streams/issues/144#issuecomment-1816534; they haven't commented on the root cause of the issue, though, so I'm not sure why those posts were sent to FediTips in the first place
• their explanation at https://codeberg.org/streams/streams/issues/144#issuecomment-1817434 contains some inaccuracies: the activities sent to your personal inbox were accepted, which is precisely why these posts were ingested by your server. Mastodon does have some relevancy check on the sharedInbox, but not on the personal inboxes, as Mastodon considers that delivering a post to your inbox is a deliberate action that allows you personally to see the post; we should probably add a relevancy check here as well, but we'd need to be careful to make sure it does not break anything else
• I'm still confused by elmussol claiming to follow you (https://codeberg.org/streams/streams/issues/144#issuecomment-1816031) but that not being reflected on your instance, and I wonder if there might be another issue causing the following states to not be properly synced between Mastodon and Streams…

[ClearlyClaire]

In particular, this line makes any activity delivered directly to a personal inbox pass the relevancy test:

mastodon/app/lib/activitypub/activity/create.rb

Line 403 in 13fb549

return true if @options[:delivered_to_account_id]

I think we should be able to just remove this line, but this might possibly cause some activities from other implementations to be wrongfully rejected…?

[FediVideos]

Ahh okay, thank you for the follow-up! I've reopened this for these loose ends to be dealt with.

I'm still confused by elmussol claiming to follow you (https://codeberg.org/streams/streams/issues/144#issuecomment-1816031) but that not being reflected on your instance, and I wonder if there might be another issue causing the following states to not be properly synced between Mastodon and Streams…

I think they do follow my account, but I have never followed (or heard of) any of them. I didn't think them following me was relevant for content appearing in my Federated timeline?

[ClearlyClaire]

I think they do follow my account, but I have never followed (or heard of) any of them.

Ah, well, I was going to double-check but you blocked them so I can't 😅

I didn't think them following me was relevant for content appearing in my Federated timeline?

It's not for Mastodon but it might be for Streams. And even if the inconsistency isn't causing this specific issue, such an inconsistency would be a significant issue on its own.

[FediVideos]

I think they do follow my account, but I have never followed (or heard of) any of them.

Ah, well, I was going to double-check but you blocked them so I can't 😅

Oh, sorry, I thought you'd finished! 😦 I'd checked with Hugo that the custom script had been removed first.

I am pretty sure they were following me because Mastodon alerted me to losing followers when I suspended their instance.

[FediVideos]

Okay, although Federated is almost silent now, a couple of new inexplicable posts have appeared:

https://wetdry.world/@fish/112439688889831458

https://hachyderm.io/@voyager/112437291516437753

They do both follow me, but they're following from Mastodon. Can't see any other connections and can't see Streams accounts in their followers.

[FediVideos]

...and another inexplicable post, this time with a reply from a Streams instance but it's an instance I'd already suspended:

https://antifa.style/@walsonde/112440037098068683

Perhaps the reply was delivered to another Streams instance which I haven't blocked, and then spammed to my instance? If so this could be an almost impossible game of whackamole to do manually 😫

Would be great if there was some barrier on Mastodon to prevent this kind of delivery.

[ClearlyClaire]

This is a pinned post for that account, which may have been pulled for any number of reasons.

[FediVideos]

Some more inexplicable posts have started appearing, they do seem to be mainly pinned posts:

https://catwithaclari.net/notes/9thh82gqazom01xk
https://eldritch.cafe/@jessienada/112467404954046170
https://subs4social.xyz/notes/9tgky9m3j0
https://me.dm/@matthewspira/112462691899273311

...but there is one non-pinned post:

https://astrodon.social/@schuh/112467770247736167

This is a pinned post for that account, which may have been pulled for any number of reasons.

I get that pinned posts are backfilled (which is great by the way! 👍 ) but I thought that only happened if I did something myself, such as me bringing up their profile on my instance, or me following them, or me interacting with them in some way?

Do pinned posts get backfilled simply from them following me, or them boosting me or them favouriting me? Even if I've never interacted with them?

[ClearlyClaire]

Pinned posts get backfilled when your server discovers about the account, for any reason (them interacting with you, them being mentioned in a post that reaches your server, and so on)

[FediVideos]

Ahhh okay. So, for example, if someone mentioned me and mentioned the other account in the same post, that would backfill the other account's pinned post and make it appear on my Federated?

[ClearlyClaire]

yes

[BillStatler]

Hi, I am the billstatler@forum.statler.ws mentioned above. I'm not a software developer, but I think I understand what is going on here.

I do have a connection with @FediVideos -- or I did, before I got blocked. My channel connected to LooseEnds@social.growyourown.services on 2024-01-01. So how did LooseEnds start following me without knowing about it?

In Streams, connections are based on a set of permissions, and this model doesn't map well onto Mastodon's follower/following model. It's more like "friending" someone on Facebook. If I send you a Facebook "friend" request, and you accept, we will see each other's posts. (Unless you take the additional step of "unfollowing" me without "unfriending" me.)

So my billstatler channel sent a connection request to LooseEnds, saying basically "Here are a bunch of permissions I will grant to you, related to seeing my posts, commenting on them, seeing my photos, etc. Will you accept, and will you grant me permissions for your posts/photos/etc?"

This request was accepted, at which point we were (in Mastodon terms) following each other. Thereafter, my server correctly sent my activity to LooseEnds. In the example posted earlier, I commented on a post by coffeegeek@flipboard.social and my comment went to all of my connections including Loose Ends, whose server then did its Mastodon magic to find the post I was commenting on.

I have never used Mastodon, so I don't know what this looks like from your end. Is there a way to see who you're following, or to verify whether you intend to follow somebody when you approve a connection request?

Anyway, blocking all Streams users isn't a good or sufficient solution. You'd probably also have to block all Hubzilla users, and maybe Friendica, and perhaps other projects that I don't know about.

The lead Streams developer has added some code to reduce unwanted deliveries, but it can't fix a situation like this where somebody is actually following a Streams user (even if they don't know it).

[ClearlyClaire]

@BillStatler hi! Mastodon uses the base ActivityStreams vocabulary and ActivityPub semantics regarding follow relationships. Basically, you send a Follow activity to another user, and they either Accept or Reject it. There is no baked-in notion of bidirectional friendship relationship, nor a way to add someone to your followers without their consent.

If I understand what you suggest, you sent a Follow request to LooseEnds@social.growyourown.services which was (presumably automatically) Accepted, at which point Streams considered there was a bidirectional connection? However, at the protocol level, such a connection is not implied.

[BillStatler]

Thanks for the reply, @ClearlyClaire

I'm afraid my technical understanding of ActivityStreams/ActivityPub is pretty limited, so I can't tell you how it became a bidirectional connection. But my instance believed that it was. I recovered some information from a backup of my billstatler channel, and here is how my connection to LooseEnds appeared as of 2024-02-25:

In this table, "Their Settings" are the permissions granted by LooseEnds to billstatler, and "My Settings" are the permissions granted by billstatler to LooseEnds. They are completely symmetrical. In particular, the "Accept delivery" permissions granted by LooseEnds are all YES, and I assume this is why my instance sent all my stuff to his instance.

Now that I have been blocked, all the "Their Settings" YES's have become NO's.

So logically, I can only see three possibilities:

• When LooseEnds approved my connection request, he followed billstatler without realizing it.
• Or, he did not follow, but his Mastodon instance and my Streams instance failed to communicate about this.
• Or, my Streams instance got the information but didn't update the permissions table. (This seems unlikely, because Streams was able to update permissions after I got blocked.)

That's the best information I can offer. I know Mike Macgirvin has made some recent changes related to deliveries, and he is working on more changes. So we may see fewer unwanted deliveries in the future. But in my particular case, I'm not convinced there was a delivery error.