Diaspora Relay Server Issue

muppeth

Wed, 22 Aug 2018 10:55:33 +0000

I've spoken yesterday with Jason Robinson (owner and author of diaspora relay server) about the possible issue of not getting diaspora posts from relay server (based on the tags or firehose). It seems like its not only my hub that is affected but all of them. Apparently we are sending our request as array while the relay server expects an object: we send "tags":{"0":"argentina","1":"uruguay", ...} instead of "tags":["argentina","uruguay",...]
It doesnt matter whether one has a firehose enabled since those request fail because of tags (unless tags are disabled).

I tried to find a fix for it last night in diaspora app, but failed so far. I will continue to try as its a nice exercise but I thought maybe others will be faster and solve it before.

show all 27 Comments

Mario Vavti

Wed, 22 Aug 2018 11:18:58 +0000

That is weird since importing tags does work for me. I'll have a look into it though...

muppeth

Wed, 22 Aug 2018 12:11:15 +0000

here is the spec for the doc i got sent from jason: https://github.com/jaywink/social-relay/blob/master/docs/relays.md#node-subscription-preferences

Mario Vavti

Wed, 22 Aug 2018 15:19:56 +0000

This is weird. My hub returns the correct json: https://hub.somaton.com/.well-known/x-social-relay
while the disroot hub returns the wrong json: https://hub.disroot.org/.well-known/x-social-relay

Mario Vavti

Wed, 22 Aug 2018 19:07:24 +0000

After some more digging it appears to be another hidden postgres issue. For some reason the result is different when using mysql.

Anyway, i have a possible fix. In diaspora/diaspora.php change line 125 from
'tags' => (($tags) ? $tags : [])
to
'tags' => (($tags) ? (array)$tags : [])
and please report if that fixes the issue.

muppeth

Sat, 25 Aug 2018 08:01:28 +0000

@Mario Vavti Thanks. It crossed my mind that that postgres could be the issue. Sweet I will change it and check with jason.

Mario Vavti

Sat, 25 Aug 2018 08:07:13 +0000

@muppeth you should be able to see the difference just by looking at the json output. You should get square instead of curly brackets for the tags field.

muppeth

Sat, 25 Aug 2018 08:24:31 +0000

@Mario Vavti dont see any difference in the ourput: https://hub.disroot.org/well-known/x-social-relay

Mario Vavti

Sat, 25 Aug 2018 10:00:44 +0000

Sigh...

Sat, 25 Aug 2018 11:18:48 +0000

https://stackoverflow.com/questions/35099682/php-array-to-json-object

Mario Vavti

Sat, 25 Aug 2018 12:21:05 +0000 last edited: Sat, 25 Aug 2018 12:30:29 +0000

But what we need for tags is an array, not an object (according to the spec):

Example .well-known/x-social-relay document:

{
  "subscribe": true,
  "scope": "tags",
  "tags": ["foo", "bar"]
}

Sat, 25 Aug 2018 22:28:49 +0000

Apparently we are sending our request as array while the relay server expects an object: we send "tags":{"0":"argentina","1":"uruguay", ...} instead of "tags":["argentina","uruguay",...]

shrug

Mario Vavti

Sun, 26 Aug 2018 03:23:12 +0000

I have set up a local test with postgres and still got the correct result. So it is actually not pg related...

Mario Vavti

Sun, 26 Aug 2018 03:45:47 +0000

@muppeth what versions of php and postgresql are you using?

Sun, 26 Aug 2018 06:09:22 +0000

    $arr = array(
            'subscribe' => (($disabled) ? false : true),
            'scope' => $scope,
            'tags' => (($tags) ? array_values($tags) : [])
        );

Mario Vavti

Sun, 26 Aug 2018 06:48:13 +0000

I wonder if this could be the culprit. From the PHP docs on serialize():

Returns a string containing a byte-stream representation of value that can be stored anywhere.

Note that this is a binary string which may include null bytes, and needs to be stored and handled as such. For example, serialize() output should generally be stored in a BLOB field in a database, rather than a CHAR or TEXT field.

Harald Eilertsen

Sun, 26 Aug 2018 07:20:50 +0000

Hm... seems hub.volse.no has this issue too. Also using pg. But this used to work. When I set up the hub, it definitely got posts from diaspora by tags. I have some other issues with the diaspora plugin that I'm still trying to find time to figure out too.

Mario Vavti

Sun, 26 Aug 2018 07:23:11 +0000

@Harald Eilertsen can you try Mike's proposed fix?

Sun, 26 Aug 2018 07:30:59 +0000

If you do a browser page search for postgres on the php serialize page there's an interesting note about postgres. It appears we'll have to translate null bytes in all the config classes as well as the session class. I really hope we don't have to worry about multibyte chars elsewhere in postgres, but that's the inference.

muppeth

Sun, 26 Aug 2018 07:33:10 +0000

@Mario Vavti i'll check it later day when i'm back at home. Though you're using postres too.

Mario Vavti

Sun, 26 Aug 2018 07:37:18 +0000

Though you're using postres too.

Yeah, but i could not reproduce this issue on my postgres install

Harald Eilertsen

Sun, 26 Aug 2018 07:47:56 +0000

@Mario Vavti @Mike Macgirvin I will try it when I get back home. Tonight or most likely tomorrow.

Sun, 26 Aug 2018 07:54:48 +0000

If there is a problem with postgres it will be with accented characters.

Mario Vavti

Sun, 26 Aug 2018 18:23:13 +0000

In fact it appears some mysql installations are also affected: https://hubzilla.eu/.well-known/x-social-relay

Sun, 26 Aug 2018 23:52:46 +0000

#fixed - will submit in a minute.

Sun, 26 Aug 2018 23:59:09 +0000

The low down explanation is that array_unique is used to build the config list and array_unique preserves the original keys and can leave holes where duplicates once were. If there are any holes in the array json_encode treats it as associative. I've also fixed the output side so that the fix will work immediately once you have the update rather than waiting for the next time the tag list is built.

Mario Vavti

Mon, 27 Aug 2018 07:58:01 +0000

I pushed the fix to master. Please pull addons and report if the issue is fixed.

Harald Eilertsen

Mon, 27 Aug 2018 09:36:18 +0000

@Mario Vavti @Mike Macgirvin Looks good:

{"subscribe":true,"scope":"tags","tags":["metal","diaspora","programming","rust","floss", ...

Excellent work, thanks!