Posts made by jimcavoli

@girish I’m still around - I can pick this back up in the next week or two

LDAP groups would be huge. Been dying for that a couple years now

Got 7.0.1 loaded up on my testbed machine...going to re-update the packaging as needed and ensure recvmail is integrated properly, then we should be good to go here. Hoping for O(days) on that, may be O(weeks) given schedules the next few weeks.

@hollosch No, work remains "in progress" for the time being to get a reliable package finished before it heads there. You can keep track by the "WIP" tag on the thread right now - it'll go "Solved" and green once completed

@nebulon Also while X-Frame-Options is not as current as CSP, it's still considered best practice to get more complete coverage for that protection across browsers, especially older ones:

https://caniuse.com/contentsecuritypolicy2
https://caniuse.com/x-frame-options

At least, that's still the case for every audit and best practice list in the circles I'm in. It is still required by the latest ASVS 4.0.2 (criteria 14.4.7) as well (source: en / de). So I'd encourage both. While you're touching the session cookie, you can also probably go SameSite=Strict as well.

@chymian said in Quo Vadis Cloudron?:

none of you, who where so quick with the answers and flames did understand, what my point is!

That's sort of exactly my point. You took a pretty condescending tone with comments like "or do you want to deliver a system, which behaves like a real server" that are just vaguely argumentative, and the whole thing led off with slamming the community for not answering certain questions to your satisfaction. You led right into your opinions of "how a server should behave" and how that isn't this product, peppered with value judgements about different technical specialties. Yes, the reaction was collectively a bit defensive, but I don't think it's a stretch to understand why.

Either way, I still struggle to understand your goal/point/complaint. It seems like perhaps you're just advocating for more configurations for ever more behaviors and included features. There are multiple ways to achieve everything you've discussed. I don't know what the hangup about 2FA is, but the feature you're complaining about is this:

Which seems to be exactly what you think it should be - a setting to require 2FA for users.

There's every choice in how someone wants to manage/monitor their servers. Nothing about Cloudron precludes installing agents for any sort of monitoring/management system whatsoever for the underlying host. Want to wrangle your k hosts with ansible? Fine. Rather prefer DataDog or Nagios or PRTG or whatever? Go for it. If the point is that Cloudron should make more decisions about things like this, I disagree. If rather you think it precludes their use somehow, it does not. If you think that it should make some of its activities more apparent for such tools to notice, I think we'd have an interesting conversation to have. If you meant something else, please explain further, because at this point those are the only parts I understand.

As a professional whatever, I read all of that, and may I just say...uh, what?

Footnote: pretty difficult to imagine "more control" than root, but that's mostly because I don't consider straight assembly into ring 0 a daily-driving sort of necessity.

I agree about Moodle, and I'm a fan of having more choices in every category on the platform generally. Canvas is the leading alternative in terms of upvotes so far, and likely to arrive sooner. Worth taking a closer look - bit overwhelming to go in through Tutor itself - I'd advise anyone looking to go underneath that (massive) installer/abstraction and check out Open edX directly as a starting point. Just .02 from a cursory look; this isn't too high on my list yet, but I'll try to make a deeper evaluation of it soon.

You can't use that particular method of editing the code, but you could look into setting a custom CSP via the app configuration (may need some maintenance to keep everything working in the future) - https://docs.cloudron.io/apps/#custom-csp - You'd need to get a copy of the CSP header being sent by your installation presently, then you can adjust the iframe sources allowed manually and put the entire adjusted CSP in place.

@girish So the gzip failure is independent the staged builds. Arguing with tar/gzip and GitHub about file formats presently, but I can replicate the issue you were having and am testing against the same Dockerfile with and without buildit on, so once I get that sorted, we can regroup and go from there.

Also running a moderately large EspoCRM instance here, plus some more hobby-scale ones, and agree with all the points made so far. Generally, I'm a fan. The Advanced pack and VoIP extensions have been good additions and I've not had any issues with the company behind EspoCRM either. Just works, admittedly with a bit of learning curve to administer, perhaps, as has been mentioned, but a lot of that is down to the customizations available. Documentation is only okay for most things, in my opinion. Overall, I'd recommend it

@girish yeah, I don't see why not. It works in an older version. I may be able to pin back the specific dependency as well (it's the node dependency from https://github.com/loomio/loomio_channel_server that seems to be doing it). I'll play around with it a bit while we work on the recvmail thing in the mean time.

@girish Yeah, I tried setting that, but the issue is that one part of the app is sending the two-argument version every time, which 5 and lower just don't know how to handle

@girish All else held equal, I'd probably take this compromise. However, the apps in question where I have run into this issue have the email interface as core features and it can't really be turned off. Installing a somewhat handicapped version of these apps which would require further manual configuration, in some cases through the terminal, is a really poor experience.

I think the case that the mail server is external to the Cloudron should be a well-accounted for exception, but not the expectation of packaged apps. My "perfect" reimagined recvmail would be that by default it works as you've described, but there would be new options on the app configuration page for "Email" that would allow switching the CLOUDRON_MAIL_IMAP_* details over to a manually-entered set of values if desired. That would keep things automatic for the all-in users and allow flexibility for the split-server case, all without requiring complex configuration changes to files and/or the apps' packaging.

These are the log messages when executing a test (which exercises both sending and receiving) through OneDev. Taken from the Cloudron-side email logs, with expanded details, domain name redacted, and chronology flipped (oldest message first):

Queued mail for delivery to onedev.app+test-sub-addressing@example.com from onedev.app@example.com

{
  "ts": 1629216044183,
  "type": "queued",
  "direction": "outbound",
  "uuid": "22C112A7-5AA8-4685-8D85-11BBE6DC4C28.1",
  "remote": {
    "ip": "172.18.16.253",
    "port": 48192,
    "host": "05b821f7-a64e-40c0-8296-451deb089e0c.cloudron",
    "info": "05b821f7-a64e-40c0-8296-451deb089e0c.cloudron",
    "closed": false,
    "is_private": true,
    "is_local": false
  },
  "authUser": "onedev.app@example.com",
  "mailFrom": "<onedev.app@example.com>",
  "rcptTo": [
    "<onedev.app+test-sub-addressing@example.com>"
  ],
  "details": {
    "spamStatus": "",
    "message": "Message Queued (22C112A7-5AA8-4685-8D85-11BBE6DC4C28.1)"
  }
}

Sent bounce to onedev.app@example.com for mail sent to onedev.app+test-sub-addressing@example.com. Some recipients failed: <onedev.app+test-sub-addressing@example.com>

{
  "ts": 1629216044307,
  "type": "bounce",
  "direction": "outbound",
  "uuid": "22C112A7-5AA8-4685-8D85-11BBE6DC4C28.1.1",
  "mailFrom": "<onedev.app@example.com>",
  "rcptTo": [
    "<onedev.app+test-sub-addressing@example.com>"
  ],
  "details": {
    "message": "Some recipients failed: <onedev.app+test-sub-addressing@example.com>",
    "mx": {
      "priority": 0,
      "exchange": "127.0.0.1",
      "port": 2424,
      "using_lmtp": true,
      "family": "A",
      "bind_helo": "mail.example.com"
    },
    "bounced_rcpt": [
      {
        "original": "<onedev.app+test-sub-addressing@example.com>",
        "original_host": "example.com",
        "host": "example.com",
        "user": "onedev.app+test-sub-addressing",
        "reason": "550 5.1.1 <onedev.app+test-sub-addressing@example.com> User doesn't exist: onedev.app@example.com",
        "dsn_action": "failed",
        "dsn_smtp_code": "550",
        "dsn_smtp_extc": "5.1.1",
        "dsn_status": "5.1.1",
        "dsn_smtp_response": "<onedev.app+test-sub-addressing@example.com> User doesn't exist: onedev.app@example.com",
        "dsn_remote_mta": "127.0.0.1"
      }
    ]
  }
}

Sent bounce to <> for mail sent to onedev.app@example.com. Some recipients failed: <onedev.app@example.com>

{
  "ts": 1629216044342,
  "type": "bounce",
  "direction": "outbound",
  "uuid": "D47C41D3-83FB-49F0-A709-5162706B0A72.1",
  "mailFrom": "<>",
  "rcptTo": [
    "<onedev.app@example.com>"
  ],
  "details": {
    "message": "Some recipients failed: <onedev.app@example.com>",
    "mx": {
      "priority": 0,
      "exchange": "127.0.0.1",
      "port": 2424,
      "using_lmtp": true,
      "family": "A",
      "bind_helo": "mail.example.com"
    },
    "bounced_rcpt": [
      {
        "original": "onedev.app@example.com",
        "user": "onedev.app",
        "original_host": "example.com",
        "host": "example.com",
        "reason": "550 5.1.1 <onedev.app@example.com> User doesn't exist: onedev.app@example.com",
        "dsn_action": "failed",
        "dsn_smtp_code": "550",
        "dsn_smtp_extc": "5.1.1",
        "dsn_status": "5.1.1",
        "dsn_smtp_response": "<onedev.app@example.com> User doesn't exist: onedev.app@example.com",
        "dsn_remote_mta": "127.0.0.1"
      }
    ]
  }
}

I first noticed this issue while packaging Loomio and ended up right back here again when I started working on OneDev as well.

So, as first described in January and confirmed as recently as last week and yesterday, this issue is still going on. Let's say we have a cloudron app called example which gets the email address example.app@example.com assigned from the Cloudron for it to use. When both sendmail and recvmail addons are enabled for it, I would expect that the behavior would be:

1. Both SMTP and IMAP credentials populate into the environment
2. The app can connect to both SMTP and IMAP servers
3. Once connected with the app's credentials, it is possible to send mail via SMTP and receive mail via IMAP

However, the observed behavior is:

1. Both SMTP and IMAP credentials populate into the environment
2. The app can connect to both SMTP and IMAP servers
3. Once connected with the app's credentials, it is possible to send mail via SMTP, but all mail sent to example.app@example.com (and any example.app+foo@example.com-style subaddresses) is hard bounced smtp;550 5.1.1

This is a surprising behavior anyway, but particularly tough for the common use case both these apps have, whereby they send email notifications which users can reply to via email as a way to interact with the service. Loomio does this for discussion threads, OneDev does it for issue and PR notifications, and there are many useful cases in which this sort of feature is useful. However, it is not possible to enable it "automatically" at least with a Cloudron app presently, using the managed addons' capabilities. This could all of course be manually wired up, but that's a clunky workaround that's blocking the process of at least these two apps toward general availability, but has also impacted my designs on two other custom apps I've been working on to run on Cloudron.

I've got no solutions or ideas on what is causing this under the hood presently (it's been a pretty busy year), but I wanted to make sure this post gets out there to identify the issue as known, and maybe some greater minds with more time can get it resolved eventually!

I've actually got OneDev nearly packaged locally. However, it is yet another app that has a need to both send and receive mail for certain features, and it is now blocked the same way Loomio is. With both recvmail and sendmail addons running, emails can be sent by the app just fine, but inbound mail bounces.

@girish Any progress on this defect?

@debossnow The password will be the same as the password on the Cloudron for login, but the "username" in this case should be the full email address of the mailbox which you'd like to sign into. This is how multiple mail domains can have mailboxes owned by one account. For example, let's say that I have a Cloudron at example.com which handles mail for example.org as well. Let's further say there is a user with the username admin and a password of secretpassword. That user could be assigned multiple mailboxes; for the sake of the example, let's say this user admin is the owner for admin@example.com, contact@example.com, and also info@example.org mailboxes. In order to sign in to a webmail, such as RoundCube or RainLoop that is attached to the Cloudron mail system, it would be necessary to use the full email address - e.g. contact@example.com or info@example.org - and the same secretpassword as the credentials. The Cloudron knows which user (and therefore which password) goes with a mailbox, but of course the webmail client does not and can only provided the intended mailbox.

If there's anything more unusual going on, the Event Log on the email administration page is likely to have some further details.

@robi said in Blink - Modern, lightweight, planet-scale link shortener:

self-hosted CDN

Multi-stage builds can go a long way - been beating this drum for a minute because it's pretty easy to do. That's a great article in general on the topic though. Also, docker-slim has come up before I think, which could be a useful tool. That said, the other key thing would be including less in the base image by default, but that is a longer conversation.

How much of https://docs.cloudron.io/troubleshooting/#unreachable-dashboard have you worked through? It does appear you are in the "Unreachable Dashboard" situation presently, though you may have a DNS issue. Knowing the results of the listed troubleshooting steps will help characterize the problem and appropriate solution(s) much better.

I've updated https://git.cloudron.io/jimcavoli/openslides-app with the latest changes for version 3.3 of Openslides. Dockerfile is still multi-staged for the time being. @girish do you want to take another pass at building as-is or should I work on making the Dockerfile straight through.

@girish Opened it up and got to updating - was reminded that the issue was the inbound email side of things, not the Dockerfile on this one...it's the OpenSlides packaging stalled on the multistage builds (which needs some more revision and I'll update soon). In any case, I've updated it to use cloudron/base:3.0.0 and Loomio at 2.8.3 (latest) locally and on my test box. The outstanding issue with inbound mail remains, plus a new, if minor, issue with Redis AUTH for the channel server - any chance we'll see Redis updated to 6.x soon?

I can work on alternatives to make the channel server Redis connection happy, but I still need help on the inbound email side of things. Sending seems fine, but replies (the inbound-to-app side) bounce immediately. I've pushed the changes for the newer versions to https://git.cloudron.io/jimcavoli/loomio-app and included the caveat(s) about the broken components in the commit message for now.

@girish On it; hopefully will have that today

I can take a look at making sure the packaging code is up to date, but it's very much working on a test instance still. Only issue was @girish building / testing - happy to work on that part of it if needed...I could make it a straight-through Dockerfile rather than BuildKit dependent - though it'll take a fairly long time to build that way, maybe that's just what we need to do?

At least for the hotmail/outlook addresses, you can check out SNDS - https://sendersupport.olc.protection.outlook.com/snds/index.aspx - to verify status and/or request unblocking if you're currently being blocked.

@jdaviescoates Loomio has priority over Zammad to me. I've been made unfortunately busy by other forces in the world lately, but trying to decipher where past Jim left off, it does appear Loomio is ready other than perhaps being re-updated to latest and whatnot, but mostly is held up by the Cloudron team from the app store right now.

Hopefully I can get back to poking at Zammad shortly

@jdaviescoates Yes, "not alpha" is very much the thing I was talking about

Certainly nothing wrong with getting it packaged unstable and so on while it's still under more active development, but my experience with how much of the critical components of an app that can break the building of a package are much more likely in alpha-land than in beta-land at least. YMMV, and words are made up with different meanings to different teams/people, especially in this one, but that's where my brain's at presently.

@girish Yeah, what @jdaviescoates posted is the thing - it's largely not documented all that well from the technical setup perspective. I just sort of...figured it out? There's not that many components and it's a pretty standard python app. Because it's not well-documented on the project's part insofar as running is concerned, that's a large part of why I made the readme and put all the notes in that I did.

You should see if they'd take a patch upstream for this! Nice work

@atrilahiji That sounds like a good approach; we do similar things in other apps for unnecessarily obtuse LDAP setups, for example.

@girish I see what you're saying. Perhaps an ability to log into one's Cloudron.io account and post via a subcommand cloudron publish with the CLI tool would be able to send a payload of the current docker repository URL and manifest to an endpoint, which could then be sanity-checked that it doesn't require authentication, and added into the app store updates/listings (optionally with a review step), marked as community-supported. The review step is probably fairly important to see that there's a support email and such listed, but we could get additional forum topics and a process for contributing to the documentation as well set up in time.

Looking the other way, there's really no reason that the Manifest can't itself specify the image repository URL, right? Then it would just be a matter of POSTing that manifest to an endpoint for the proper delivery to be possible downstream. Opens up a lot of options the more that I think about it. Sort of an elegant way to do continuous delivery automation for any custom apps as well.

Maybe it's because of the brilliant person who posted the request, but it strikes me that mutual TLS optionally and globally on the frontside reverse proxy is a more elegant way to achieve a similar result: https://forum.cloudron.io/topic/3826/support-optional-global-https-mutual-tls-certificate-based-authentication

This is on my radar for packaging, though at the moment I'm holding off until the codebase is more stable, unless consensus build that packaging sooner is a good idea

You can edit/remove the information in $HOME/.cloudron.json that deals with that folder on your machine to correct the repository info or just delete the block to make the CLI re-prompt you on the next run.

@jdaviescoates Looks really neat, except for the fact it's built on Drupal...that aside, shouldn't be too tough to package. I'm actually surprised nobody's done Drupal yet separately

This would actually explain a whole class of strange behavior I've seen with jobs getting backlogged when running more frequently than 10min.

Yeah, you likely don't want localhost you want the uuid of the installation as the hostname since that's what's exposed in the docker virtual network.

@hillside502 That's a really good note - the underlying opensource project is MIT licensed though: https://github.com/kerberos-io/opensource/blob/master/LICENSE

@atrilahiji I'm increasingly inclined to agree - and the Greenlight interface being available and usable by Cloudron LDAP seems like the best of all worlds for that split. That's a really elegant solution that meets needs in my opinion for anyone who needs heavier-duty conferencing.

A few notes would be that I agree that this is likely not a huge priority insofar as protecting the on-disk data, but it would be a nice add. That said, to the point about using passwords as keys, that's a hard no - aside from the password-changing problems, it's recognized as a Bad Idea by the security community:

Verify that the architecture treats client-side secrets--such as symmetric keys, passwords, or API tokens--as insecure and never uses them to protect or access sensitive data.

---

Source: https://github.com/OWASP/ASVS/blob/v4.0.2/4.0/en/0x10-V1-Architecture.md#v16-cryptographic-architectural-requirements

k8s is not a great fit imo for cloudron without introducing much bigger changes...there are roads to that runtime with some intermediary schedulers as well though, which is why I like Nomad in this space the most. I've actually been working up a prototype using the HashiStack Consul/Nomad (plus or minus vault) to provide a distributed runtime, but that's a reasonably long way off seeing any sort of integration into the core of things. It's a big shift on its own, and needs a lot of refinement. Obviously so would a k8s approach. In the immediate term, managing across multiple full-on cloudron instances is fairly clean, and if implemented correctly, could actually still be useful in that world as well. It's the first, easiest, smallest thing to do and therefore in my opinion is valuable, regardless of where the higher-powered distributed runtime ideas go.

@girish said in Akaunting - Free Accounting Software:

Thanks for helping us out @denisdulici ! The work is happening here - https://git.cloudron.io/cloudron/akaunting-app/

Dead link; status seems unknown. Have you got an update on the old effort and/or would a new pass be more useful?

That should all be getting logrotate-d to some extent, depending on the subdirectory. Configs are in /home/yellowtent/platformdata/logrotate.d if you want to have a peek but generally those should be getting rotated off pretty fast. A couple gigs isn't super unusual in my experience, but the bulk of that is likely app logs. AFAIK, there's not really any harm to dropping out logs other than the fact that you won't have them, but have a look at the breakdown inside the logs directory - all the logs/{UUID} folders are the app ones and likely of only marginal utility to keep, depending on your needs.

@girish To that end, perhaps this one is better locked at this point

Yeah, wherever I need yarn, I'm installing it via npm and often actually installing particular node versions directly as well in my packagings. IMO we're reaching a scale of seriously diminishing returns, and frequently apps now are including the versions they need in the actual source in asdf .tool-versions or similar files. I specifically try to automate the runtime selection/installation at build time when this information is available from the app - it increases compatibility to be more specific, if taking a hit on container bloat since it often doubles up on base image contents.

Yeah, I think I misinterpreted your question a little - glad you're all set though!

@jordanurbs That looks an issue with the TLS implementation underpinning the mail function using peer matching that is too strict/dumb on the hostname. They're using a wildcard cert and for whatever reason PHP is balking at the * wildcard and expecting that to literally be the hostname. This was a known bug in PHP like 7 years ago, but shouldn't be a contemporary problem. What version of PHP are you using?

Honestly, I'd like to see GitLab EE on Cloudron as well, but my hangup with whether to package it or not has a lot more to do with not handicapping the more advanced features of the EE server especially when compared to the officially supported variants with either omnibus or kubernetes deployment. Frankly, it's heading more and more Kubernetes-focused for runtime, so it's likely not worth it, but I'm open to changing my mind. I haven't tried to package it and see what happens and likely won't without sponsorship given the cost of even a disposable Ultimate license.

@atrilahiji Just you in the sense that I didn't run into that when updating any of mine. When I have previously, it's usually been a resource contention thing that cleared up running at a lower-load time, but it's been several versions (and hardware bumps) since I've seen it come up.

@robi Could well be - also could be something underlying in their sanitizing. I've had a few instances getting absolutely hammered from Germany on contact forms, mostly getting blocked by recaptcha but ended up being more of a DOS for the resources they threw at it. Could be the same thing, might not be, but I've cut them off early and low in the stack, so logging/etc. is pretty minimal at this point since the firewall is dropping them. FWIW, the contact forms are Caldera - not sure if that's in common or not, but that's a pretty broad attack surface to start from if so.

@marcusquinn That statement could be applied to at least 75% of things about Nextcloud.

@subven Totally agree - by that logic, they should ban all the web browsers because they could connect to a site with highly objectionable content. Probably all the VPNs and other general-purpose utilities too. It's a dangerously slippery slope for sure.

@lonk That's a totally valid, effective way to do it. Probably the most instantaneous of all the things mentioned, since it seems speed/latency is a priority for you. Lambda's not that bad to write code for - if you get pretty advanced with it, you can really run nearly any language on the platform, though the officially supported Java, Go, PowerShell, Node.js, C#, Python, and Ruby are the easiest options. You've got a lot of choices there.

Honestly, this will mostly depend on which two apps you want to run. There's a huge spectrum of actual RAM required based on which apps and how many users you want to support on it.

This should likely get the same fix as the Metabase package did to make the JVM limits better reflect the available resources.

Context: https://forum.cloudron.io/topic/3588/frequent-java-lang-outofmemoryerror-java-heap-space-errors/5?_=1612137576983
Fix (for Metabase): https://git.cloudron.io/cloudron/metabase-app/-/blob/master/start.sh#L38-42

cc @girish

I really like the blackllist checking being built-in. Frankly, I'd also be a fan of getting notifications about it. I suppose UX-wise, perhaps this is the appropriate sort of thing to trigger a yellow status on email.

@robi's post made good point that just made me realize - I assumed from your question you are looking for a full mailbox that would receive mail and the user would log into SOGo to access. That is what my answer applies to.

In a word, no, that's not supported. As the app depends on the mail server's implementation in general, and Cloudron's does not support this, neither does SOGo.