<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[GET GROUP truncated]]></title><description><![CDATA[<p dir="auto">My attempts to get a group with many members via the API seems to truncate the number of members returned at 1599 [and the 1599th member uid is truncated mid-string]. Is there a way to paginate a group or groups GET or return the full membership?</p>
]]></description><link>https://forum.cloudron.io/topic/12588/get-group-truncated</link><generator>RSS for Node</generator><lastBuildDate>Tue, 06 Oct 2026 12:54:14 GMT</lastBuildDate><atom:link href="https://forum.cloudron.io/topic/12588.rss" rel="self" type="application/rss+xml"/><pubDate>Sat, 05 Oct 2024 18:12:36 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to GET GROUP truncated on Fri, 11 Oct 2024 19:41:27 GMT]]></title><description><![CDATA[<p dir="auto">Thank you!</p>
]]></description><link>https://forum.cloudron.io/post/95532</link><guid isPermaLink="true">https://forum.cloudron.io/post/95532</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Fri, 11 Oct 2024 19:41:27 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Wed, 09 Oct 2024 17:16:54 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/darren" aria-label="Profile: darren">@<bdi>darren</bdi></a> thanks! I have bumped the limit now. <a href="https://git.cloudron.io/cloudron/box/-/commit/34969d9980da38937f88201a01460109f04a60ac" target="_blank" rel="noopener noreferrer nofollow ugc">https://git.cloudron.io/cloudron/box/-/commit/34969d9980da38937f88201a01460109f04a60ac</a> . Guess we have to make member pagination API at some point.</p>
]]></description><link>https://forum.cloudron.io/post/95406</link><guid isPermaLink="true">https://forum.cloudron.io/post/95406</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Wed, 09 Oct 2024 17:16:54 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 19:56:42 GMT]]></title><description><![CDATA[<p dir="auto">Ah I think I've found it. You're setting <code>group_concat_max_len = 65536</code> on <a href="https://git.cloudron.io/cloudron/box/-/blob/master/src/database.js" target="_blank" rel="noopener noreferrer nofollow ugc">line 66 of database.js</a>.<br />
Looks like this commit: <a href="https://git.cloudron.io/cloudron/box/-/commit/94a196bfa00ff689b0862fe797d64a917111a91d" target="_blank" rel="noopener noreferrer nofollow ugc">https://git.cloudron.io/cloudron/box/-/commit/94a196bfa00ff689b0862fe797d64a917111a91d</a></p>
]]></description><link>https://forum.cloudron.io/post/95366</link><guid isPermaLink="true">https://forum.cloudron.io/post/95366</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Tue, 08 Oct 2024 19:56:42 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 19:24:17 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/girish" aria-label="Profile: girish">@<bdi>girish</bdi></a> Good news I was able to isolate a little more. It seems like 65536 bytes is the length at which the group_concat was actually being truncated. So setting <code>group_concat_max_len = 65536</code> wasn't doing anything. But if I set that to a sufficiently higher value, e.g. <code>group_concat_max_len = 4294967295</code>, we can get the full group membership within mysql. I have not yet been able to get the full response from the API though.</p>
<pre><code>mysql&gt; SHOW VARIABLES LIKE '%group_concat%';+----------------------+------------+
| Variable_name        | Value      |
+----------------------+------------+
| group_concat_max_len | 4294967295 |
+----------------------+------------+
1 row in set (0.01 sec)

mysql&gt; SELECT length(GROUP_CONCAT(groupMembers.userId))-length(replace(GROUP_CONCAT(groupMembers.userId),',','')) AS userIds FROM userGroups LEFT OUTER JOIN groupMembers ON userGroups.id = groupMembers.groupId WHERE userGroups.id = 'gid-4802e498-4b29-4eec-bcee-57186b126aa4';
+---------+
| userIds |
+---------+
|    6047 |
+---------+
1 row in set (0.01 sec)
</code></pre>
]]></description><link>https://forum.cloudron.io/post/95365</link><guid isPermaLink="true">https://forum.cloudron.io/post/95365</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Tue, 08 Oct 2024 19:24:17 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 16:24:49 GMT]]></title><description><![CDATA[<p dir="auto">mmm, I was pretty sure it's the group concat.</p>
<p dir="auto">You can try the mysql directly. <code>mysql -uroot -ppassword box</code>. and then (just replace the group id below),</p>
<pre><code>mysql&gt; SELECT id,name,source,GROUP_CONCAT(groupMembers.userId) AS userIds FROM userGroups LEFT OUTER JOIN groupMembers ON userGroups.id = groupMembers.groupId WHERE userGroups.id = 'gid-07e5c661-f43b-45b1-8dc0-8355ca3f05da' GROUP BY userGroups.id
    -&gt; ;
+------------------------------------------+--------+--------+-----------------------------------------------------------------------------------+
| id                                       | name   | source | userIds                                                                           |
+------------------------------------------+--------+--------+-----------------------------------------------------------------------------------+
| gid-07e5c661-f43b-45b1-8dc0-8355ca3f05da | people |        | uid-1b9e3dcd-d91d-462b-8dac-d13f01febb14,uid-ca8ccd80-6cd2-4221-9150-e0b2d69333f4 |
+------------------------------------------+--------+--------+-----------------------------------------------------------------------------------+
1 row in set (0.00 sec)

</code></pre>
]]></description><link>https://forum.cloudron.io/post/95360</link><guid isPermaLink="true">https://forum.cloudron.io/post/95360</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Tue, 08 Oct 2024 16:24:49 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 15:43:14 GMT]]></title><description><![CDATA[<p dir="auto">Hmm nm I looked at <code>max_allowed_packet</code> and presumably the default of 64MB is not constraining the response.</p>
]]></description><link>https://forum.cloudron.io/post/95356</link><guid isPermaLink="true">https://forum.cloudron.io/post/95356</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Tue, 08 Oct 2024 15:43:14 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 15:41:12 GMT]]></title><description><![CDATA[<p dir="auto">Unfortunately the response is the same so far after updating the config file and restarting mysql on the server. I am going to try set <code>max_allowed_packet</code> to a larger value as well.</p>
]]></description><link>https://forum.cloudron.io/post/95355</link><guid isPermaLink="true">https://forum.cloudron.io/post/95355</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Tue, 08 Oct 2024 15:41:12 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 15:31:18 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/darren" aria-label="Profile: darren">@<bdi>darren</bdi></a> I think the issue is that we use GROUP_CONCAT and apparently this has a max length - <a href="https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_group_concat_max_len" target="_blank" rel="noopener noreferrer nofollow ugc">https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_group_concat_max_len</a></p>
<p dir="auto">Can you try editing <code>/etc/mysql/mysql.cnf</code> and adding <code>group_concat_max_len=65536</code> (just something large) in the mysqld section ? After that <code>systemctl restart mysql</code></p>
]]></description><link>https://forum.cloudron.io/post/95353</link><guid isPermaLink="true">https://forum.cloudron.io/post/95353</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Tue, 08 Oct 2024 15:31:18 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 15:24:09 GMT]]></title><description><![CDATA[<p dir="auto">Yes it's truncated with a simple curl as well. What's weird is that the truncation also occurs within the response for the specific group at the <code>/groups</code> endpoint and not just the <code>groups/:groupId</code> endpoint.</p>
<p dir="auto">the deletion dialog has the understated member count:</p>
<blockquote>
<p dir="auto">This group still has 1599 member(s). Are you sure this group is not used?</p>
</blockquote>
]]></description><link>https://forum.cloudron.io/post/95351</link><guid isPermaLink="true">https://forum.cloudron.io/post/95351</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Tue, 08 Oct 2024 15:24:09 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 07:59:30 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/darren" aria-label="Profile: darren">@<bdi>darren</bdi></a> From a quick review of our code, it seems we don't use that API in the frontend. Only place is when you try to delete a group in the UI, the api is called to get the group count. So, if you click delete Group in the UI, do you see the correct count in the delete confirm dialog? Of course, don't follow through with the deletion (be careful!)</p>
]]></description><link>https://forum.cloudron.io/post/95331</link><guid isPermaLink="true">https://forum.cloudron.io/post/95331</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Tue, 08 Oct 2024 07:59:30 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Tue, 08 Oct 2024 07:58:17 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/darren" aria-label="Profile: darren">@<bdi>darren</bdi></a> thanks. I wonder if the issue here is that we hit some packet limits . Did you already do a simple curl to check if the output is truncated or not? i.e a simple curl and not via another app, just to eliminate all other possible issues.</p>
]]></description><link>https://forum.cloudron.io/post/95330</link><guid isPermaLink="true">https://forum.cloudron.io/post/95330</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Tue, 08 Oct 2024 07:58:17 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 15:47:36 GMT]]></title><description><![CDATA[<p dir="auto">+----------+<br />
| count(*) |<br />
+----------+<br />
|     6048 |<br />
+----------+</p>
]]></description><link>https://forum.cloudron.io/post/95289</link><guid isPermaLink="true">https://forum.cloudron.io/post/95289</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Mon, 07 Oct 2024 15:47:36 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 15:27:55 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/darren" aria-label="Profile: darren">@<bdi>darren</bdi></a> try this on the server</p>
<pre><code>mysql -uroot -ppassword -e 'select count(*) from box.groupMembers WHERE groupId="gid-4802e498-4b29-4eec-bcee-57186b126aa4"'
</code></pre>
]]></description><link>https://forum.cloudron.io/post/95287</link><guid isPermaLink="true">https://forum.cloudron.io/post/95287</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Mon, 07 Oct 2024 15:27:55 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 15:19:36 GMT]]></title><description><![CDATA[<p dir="auto">I was wrong. The UI also has truncated group membership. Here's the console:<br />
<img src="/assets/uploads/files/1728314352452-1c48770f-7111-4af4-8bf6-7818f046bed9-image.png" alt="image.png" class="img-markdown  img-fluid" /></p>
]]></description><link>https://forum.cloudron.io/post/95286</link><guid isPermaLink="true">https://forum.cloudron.io/post/95286</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Mon, 07 Oct 2024 15:19:36 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 15:00:09 GMT]]></title><description><![CDATA[<p dir="auto">Since the UI is using the very same REST api, looks like something else in your stack is truncating the response then.</p>
]]></description><link>https://forum.cloudron.io/post/95283</link><guid isPermaLink="true">https://forum.cloudron.io/post/95283</guid><dc:creator><![CDATA[nebulon]]></dc:creator><pubDate>Mon, 07 Oct 2024 15:00:09 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 14:33:47 GMT]]></title><description><![CDATA[<p dir="auto">Yes both /api/v1/groups/:groupId and /api/v1/groups/ produce the same behavior.<br />
I don't see any errors in the NGINX log</p>
<p dir="auto">here's a relevant line for a GET</p>
<pre><code>[no timestamp]  - - [06/Oct/2024:23:59:19 +0000] "GET /api/v1/groups/gid-4802e498-4b29-4eec-bcee-57186b126aa4 HTTP/1.1" 200 36890 "-" "axios/1.7.4"
</code></pre>
<p dir="auto">And the UI does show all group members</p>
]]></description><link>https://forum.cloudron.io/post/95279</link><guid isPermaLink="true">https://forum.cloudron.io/post/95279</guid><dc:creator><![CDATA[darren]]></dc:creator><pubDate>Mon, 07 Oct 2024 14:33:47 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Mon, 07 Oct 2024 10:34:29 GMT]]></title><description><![CDATA[<p dir="auto">Does the UI show all the members? If it does, then the bug is somewhere else.</p>
]]></description><link>https://forum.cloudron.io/post/95256</link><guid isPermaLink="true">https://forum.cloudron.io/post/95256</guid><dc:creator><![CDATA[girish]]></dc:creator><pubDate>Mon, 07 Oct 2024 10:34:29 GMT</pubDate></item><item><title><![CDATA[Reply to GET GROUP truncated on Sun, 06 Oct 2024 14:34:42 GMT]]></title><description><![CDATA[<p dir="auto">I guess you mean the GET <code>/api/v1/groups/:groupId</code> api route? <a href="https://docs.cloudron.io/api.html#tag/Groups/operation/getGroupById" target="_blank" rel="noopener noreferrer nofollow ugc">https://docs.cloudron.io/api.html#tag/Groups/operation/getGroupById</a></p>
<p dir="auto">In code there is no limit or pagination for the members as such, so the truncating of of the response seems to be somewhere maybe in the reverse proxy or otherwise. Do you see any errors maybe in the nginx reverse proxy logs on the system?</p>
]]></description><link>https://forum.cloudron.io/post/95225</link><guid isPermaLink="true">https://forum.cloudron.io/post/95225</guid><dc:creator><![CDATA[nebulon]]></dc:creator><pubDate>Sun, 06 Oct 2024 14:34:42 GMT</pubDate></item></channel></rss>