Using only EXTERNAL_HOST results in:
```
CRITICAL: Error subscribing to Zulips: Account is not associated with this subdomain
```
Since system bots like the nagios bot are created in
`SYSTEM_BOT_REALM`, default to that subdomain of `EXTERNAL_HOST`.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.
Now, we will soft reactivate if the user group mentions has
less than 12 members.
We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.
Fixes part of #27586.
Fixes an import cycle that breaks mypy inference with django-stubs:
zproject.settings → zproject.configured_settings →
zproject.default_settings → zerver.models.users → django.conf →
zproject.settings
(The Django settings system is really not an appropriate place to put
functions.)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes#28621
Till now, this was actually a flat 50GB despite what the /plans/ page
says and was adjusted flexibly when somebody asked for a higher limit.
This actually implements the advertised formula, but changing it to
5GB/user since that's a more reasonable limit.
Keeps the 50GB limit for sponsored Standard Free organizations and also
places it as the floor for the quota for paid orgs, to not lower this
for tiny orgs with less than 5 users.
See https://chat.zulip.org/#narrow/stream/6-frontend/topic/typing.20notifications.20efficiency/near/1664991.
As detailed in that discussion,
`TYPING_STARTED_EXPIRY_PERIOD_MILLISECONDS` and
`TYPING_STARTED_WAIT_PERIOD_MILLISECONDS` are coupled constants, and
the impact of them being large is mainly that if a user closes their
computer or loses network in the middle of typing something (not
exactly a common occasion), then the client will suggest they kept on
typing longer than they in fact did.
There's a substantial decrease in resources consumed by this feature
associated with raising `TYPING_STARTED_WAIT_PERIOD_MILLISECONDS`, so
that at least seems worth doing.
Meanwhile, because TYPING_STOPPED_WAIT_PERIOD_MILLISECONDS measures
how long we should wait before deciding to stop suggesting a user is
still typing if they were previously typing a message but paused doing
so without closing the compose box (example causes being stepping away
from the computer, tabbing to go look something up, or just thinking
for a bit).
On the one hand, even the original 5 seconds is a fairly long time to
pause to think without touching the keyboard; on the other hand,
sitting with text you've written in the compose box is likely still a
quite high intent-to-send-soon state. Increasing this to 12 seconds
seems like a reasonable balance between being too trigger-happy here
here and avoiding someone who left their computer appearing like they
are still typing for a long time afterwards.
Currently, the sender names for outgoing emails sent by Zulip
are hardcoded. It should be configurable for self-hosted systems.
This commit makes the 'Zulip' part a variable in the following
email sender names: 'Zulip Account Security', 'Zulip Digest',
and 'Zulip Notifications' by introducing a settings variable
'SERVICE_NAME' with the default value as f"{EXTERNAL_HOST} Zulip".
Fixes: #23857
The presence and user status update events are only sent to accessible
users, i.e. guests do not receive presence and user status updates for
users they cannot access.
This makes it possible to send notifications to more than one app ID
from the same server: for example, the main Zulip mobile app and the
new Flutter-based app, which has a separate app ID for use through its
beta period so that it can be installed alongside the existing app.
Originally, this was how the notification emails worked, but that was changed
in 797a7ef97b, with this old behavior
available as an option.
The footer and from address of emails that are sent when this
setting is set to True are confusing, especially when more people
are involved in a stream and since we have changed the way we send
emails, it should be removed. It’s also not widely used.
Fixes#26609.
Adds typing notification constants to the response given by
`POST /register`. Until now, these were hardcoded by clients
based on the documentation for implementing typing notifications
in the main endpoint description for `api/set-typing-status`.
This change also reflects updating the web-app frontend code
to use the new constants from the register response.
Co-authored-by: Samuel Kabuya <samuel.mwangikabuya@kibo.school>
Co-authored-by: Wilhelmina Asante <wilhelmina.asante@kibo.school>
Uploads are well-positioned to use S3's "intelligent tiering" storage
class. Add a setting to let uploaded files to declare their desired
storage class at upload time, and document how to move existing files
to the same storage class.
This in-progress feature was started in 2018 and hasn't
been worked on much since. It's already in a broken state,
which makes it hard to iterate on the existing search bar
since it's hard to know how those changes will affect search
pills.
We do still want to add search pills eventually, and when
we work on that, we can refer to this diff to readd the
changes back.
This gives more flexibility on a server with multiple organizations and
SAML IdPs. Such a server can have some organizations handled by IdPs
with SLO set up, and some without it set up. In such a scenario, having
a generic True/False server-wide setting is insufficient and instead
being able to specify the IdPs/orgs for SLO is needed.
Closes#20084
This is the flow that this implements:
1. A logged-in user clicks "Logout".
2. If they didn't auth via SAML, just do normal logout. Otherwise:
3. Form a LogoutRequest and redirect the user to
https://idp.example.com/slo-endpoint?SAMLRequest=<LogoutRequest here>
4. The IdP validates the LogoutRequest, terminates its own user session
and redirects the user to
https://thezuliporg.example.com/complete/saml/?SAMLRequest=<LogoutResponse>
with the appropriate LogoutResponse. In case of failure, the
LogoutResponse is expected to express that.
5. Zulip validates the LogoutResponse and if the response is a success
response, it executes the regular Zulip logout and the full flow is
finished.
This implements the core of the rewrite described in:
For the backend data model for UserPresence to one that supports much
more efficient queries and is more correct around handling of multiple
clients. The main loss of functionality is that we no longer track
which Client sent presence data (so we will no longer be able to say
using UserPresence "the user was last online on their desktop 15
minutes ago, but was online with their phone 3 minutes ago"). If we
consider that information important for the occasional investigation
query, we have can construct that answer data via UserActivity
already. It's not worth making Presence much more expensive/complex
to support it.
For slim_presence clients, this sends the same data format we sent
before, albeit with less complexity involved in constructing it. Note
that we at present will always send both last_active_time and
last_connected_time; we may revisit that in the future.
This commit doesn't include the finalizing migration, which drops the
UserPresenceOld table.
The way to deploy is to start the backfill migration with the server
down and then start the server *without* the user_presence queue worker,
to let the migration finish without having new data interfering with it.
Once the migration is done, the queue worker can be started, leading to
the presence data catching up to the current state as the queue worker
goes over the queued up events and updating the UserPresence table.
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors. However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors. Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.
Add a client-side Javascript sentry integration. To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry. Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
We're changing the ping interval from 50s to 60s, because that's what
the mobile apps have hardcoded currently, and backwards-compatibility
is more important there than the web app's previously hardcoded 50s.
For PRESENCE_PING_INTERVAL_SECS, the previous value hardcoded in both
clients was 140s, selected as "plenty of network/other latency more
than 2 x ACTIVE_PING_INTERVAL_MS". This is a pretty aggressive value;
even a single request being missed or 500ing can result in a user
appearing offline incorrectly. (There's a lag of up to one full ping
interval between when the other client checks in and when you check
in, and so we'll be at almost 2 ping intervals when you issue your
next request that might get an updated connection time from that
user).
To increase failure tolerance, we want to change the offline
threshhold from 2 x ACTIVE_PING_INTERVAL + 20s to 3 x
ACTIVE_PING_INTERVAL + 20s, aka 140s => 200s, to be more robust to
temporary failures causing us to display other users as offline.
Since the mobile apps currently have 140s and 60s hardcoded, it should
be safe to make this particular change; the mobile apps will just
remain more aggressive than the web app in marking users offline until
it uses the new API parameters.
The end result in that Zulip will be slightly less aggressive at
marking other users as offline if they go off the Internet. We will
likely be able to tune ACTIVE_PING_INTERVAL downwards once #16381 and
its follow-ups are completed, because it'll likely make these requests
much cheaper.
As of the previous commit, the server provides these parameters in
page_params. The defaults match the values hard-coded in the webapp so
far - so get rid of the hard-coded values in favor of taking them from
page_params.
This old 300s value was meaningfully used in 2 places:
1. In the do_change_user_settings presence_enabled codepath when turning
a user invisible. It doesn't matter there, 140s is just since the
point is to make clients see this user as offline. And 140s is the
threshold used by clients (see the presence.js constant).
2. For calculating whether to set "offline" "status" in
result["presence"]["aggregated"] in get_presence_backend. It's fine
for this to become 140s, since clients shouldn't be looking at the
status value anymore anyway and just do their calculation based on
the timestamps.
Alex Vandiver
Prakhar Pratyush
Prakhar Pratyush
Anders Kaseorg
Mateusz Mandera
Aman Agrawal
Tim Abbott
Sahil Batra
Greg Price
Esther Anierobi
Simon Michalke
Samuel
evykassirer