Commit Graph

1272 Commits

Author SHA1 Message Date
Tim Abbott ec4221abb9 actions: Return message ID from internal_send_*_message.
This is handy for code that needs to do something with the sent
message.  We need it for a retention policy code path, but it seems
likely we'll use it a lot down the line.
2019-05-19 19:18:06 -07:00
Roman Godov a50824e031 models: Rename Subscription.in_home_view field to is_muted.
This renames Subscription.in_home_view field to is_muted, for greater
clarity as to what it does just from seeing the setting name, without
having to look it up.

Also disabled an obsolete test_migrations test.

Fixes #10042.
2019-05-12 22:08:10 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Rishi Gupta d2b32749cc audit log: Log acting_user for do_deactivate_realm.
Currently there's no way to tell the difference between "a server admin
deactivated a realm due to it being spammy" vs "a realm admin deactivated
the realm".
2019-05-08 15:09:48 -07:00
Yashashvi Dave 02a1b2e26b server events: Add realm-time-sync event to update stream privacy.
Add event to update stream settings whenever stream privacy is changed
accordingly.

Fixes #9470
2019-05-07 17:33:59 -07:00
Anders Kaseorg 9efda71a4b get_realm: raise DoesNotExist instead of returning None.
This makes the implementation of `get_realm` consistent with its
declared return type of `Realm` rather than `Optional[Realm]`.

Fixes #12263.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-06 21:58:16 -07:00
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
clarammdantas 85ddd73d8c message_edits: Notify subscribers when a message is changed.
Extend the list of users that have to be notified when a message is
changed, so that in addition to users who have a UserMessage row, any
users who subscribed later to a stream with history public to
subscribers will also get the update.

Fixes: #8750.
2019-04-05 17:06:24 -07:00
Tim Abbott 3da06fecd5 invite: Fix validation of referred_by field.
Previously, we could 500 if an organization administrator scanned
possible PreregistrationUser IDs looking for a valid invitation they
can interact with.

They couldn't do anything, so no security issue, but this fixes that
case to just be a 400 error as it should be.
2019-03-21 16:31:18 -07:00
Tim Abbott 1599625328 mypy: Add an explicit assert to check_message. 2019-03-21 16:31:18 -07:00
Mateusz Mandera e32c444ecf email_mirror: Move some helper functions out of actions.py.
These functions don't really belong in actions.py, so we move them out,
into email_mirror_helpers.py. They can't go directly into
email_mirror.py or we'd get circular imports resulting in ImportError.
2019-03-21 15:25:57 -07:00
Rishi Gupta 6c2c351894 onboarding: Update initial messages and stream descriptions. 2019-03-21 12:34:24 -07:00
Rishi Gupta e8741c448d refactoring: Move set_default_streams into do_create_realm.
After the commits leading up to this, the only meaningful use of this
function was in the realm creation process.
2019-03-21 12:33:19 -07:00
Tim Abbott e7cc246d27 actions: Add docstring for do_update_message.
This should help with its readability.
2019-03-20 12:48:47 -07:00
neiljp (Neil Pilgrim) 7b3a9f34b7 mypy: Assert if we do_update_message then we have rendered_content. 2019-03-20 12:48:47 -07:00
neiljp (Neil Pilgrim) a80617bc70 mypy: Fix color_map default parameter for Optional. 2019-03-20 12:48:43 -07:00
neiljp (Neil Pilgrim) a40a0312f1 mypy: Correct default=None parameter typing in actions.py. 2019-03-20 12:41:34 -07:00
neiljp (Neil Pilgrim) 3d9c4494f5 mypy: Extract addressee.stream_name() check to satisfy mypy. 2019-03-20 12:41:34 -07:00
Tim Abbott 4f2c9f46bd actions: Improve format for delete events.
There were several problems with the old format:

* The sender was not necessarily the sender; it was the person who did
  the deletion (which could be an organization administrator)
* It didn't include the ID of the sender, just the email address.
* It didn't include the recipient ID, instead having a semi-malformed
  recipient_type_id under the weird name recipient_user_ids.

Since nothing was relying on the old behavior, we can just fix the
event structure.
2019-03-19 23:16:03 -07:00
Mateusz Mandera 5b86734178 email_mirror: Change stream name encoding in mirror addresses.
Fixes #9840.

Old addresses caused bugs in some cases with non-latin characters in
stream names (see issue number above). We switch to using django's
slugify helper function to convert stream names to full ascii, while
also getting rid of problematic non-alphanumeric characters, in a
reasonable way. See Django's documentation for slugify to see more about
how this function works.

Tests extended by tabbott to cover cases where we do end up with ascii.
2019-03-18 11:14:22 -07:00
Mateusz Mandera a64a075ff1 email_mirror: Ignore stream_name part of receiving address.
To prepare for changing how the stream name gets encoded into mirror
email addresses while making sure old addresses keep working, we ignore
the stream_name part when receiving emails into the mirror and we only
look at the email_token to identify into which stream to mirror the
email.
2019-03-18 11:06:51 -07:00
Tim Abbott 50dc317466 notifications: Rename notifications.py to email_notifications.py.
This library is entirely about email notifications specifically, and
this rename should help make the codebase more readable.
2019-03-15 11:02:17 -07:00
Tim Abbott 28a4c143c8 emails: Move clear_scheduled_*emails to send_email.py.
It more properly belongs as part of our outgoing email library, not
notifications.py (which is really about email notifications).
2019-03-15 11:02:17 -07:00
Raymond Akornor 89351cdd19 send_email: Add ScheduledEmail support for multiple recipients.
Follow up on 92dc363. This modifies the ScheduledEmail model
and send_future_email to properly support multiple recipients.

Tweaked by tabbott to add some useful explanatory comments and fix
issues with the migration.
2019-03-15 11:02:12 -07:00
Challa Venkata Raghava Reddy b69aec2dbc streams: Add first_message_id tracking first message in stream.
This field is primarily intended to support avoiding displaying the
"more topics" feature in new organizations and streams, where we might
know that all messages in the stream are already available in the
browser.

Based on original work by Roman Godov, and significantly modified by
tabbott.

The second migration involved here could be expensive on Zulip Cloud,
but is unlikely to be an issue on other servers.
2019-03-11 13:30:49 -07:00
Tim Abbott dbf7f8e193 zephyr: Fix handling of account deactivation.
When a Zephyr user deactivates their account, they should be
automatically turned into a mirror dummy user (so that other users can
continue to interact with them as normal for a Zephyr user who isn't
using Zulip).
2019-03-07 20:41:39 -08:00
Vishnu Ks a288cfc43a uploads: Show used upload space in attachments UI. 2019-03-07 20:18:00 -08:00
Tim Abbott 051bbf9f35 docs: Add some more links to new soft deactivation documentation. 2019-03-07 17:48:54 -08:00
Mateusz Mandera dbff533e09 email_mirror: Add the sender at the start of stream message.
Fixes part 3 of #10612. When sending an email to the email mirror to a
stream address, if "+show-sender" is added in the address, the stream
message will now include "From: <sender>" at the top.
2019-03-07 14:28:33 -08:00
Hemanth V. Alluri ae126c452b stream-descriptions: Create wrapper for rendering stream descriptions.
In commit de65a04 we can see that if the need ever arises to modify
how stream descriptions are rendered, we would need to make changes
at 5 different call points which can be quite cumbersome. So this
functionality has been extracted to a new method called
'render_stream_descriptions'.
2019-03-06 17:16:14 -08:00
Vishnu Ks 4eb5b9ad80 realm: Restrict upload quota according to plan. 2019-03-04 18:46:13 -08:00
Bennet Sunder 7c5f316cb8 alert_words: Performance improvements in looking for alert_words.
This commit leverages the ahocorasick algorithm to build a set of user_ids
that have their alert_words present in the message. It runs in linear time
of the order of length of the input message as opposed to number of
alert_words. This is after building a ahocorasick Automaton which runs
in O(number of alert_words in entire realm) which is usually cached.
2019-03-01 15:36:39 -08:00
Eeshan Garg 9252c43225 api/streams: Support including bot owner's subscriptions.
This is important for situations such as with our Zapier app,
where the requesting user may be a bot that would like to access
its owner's subscriptions.

Tweaked by tabbott to eliminate the 2^N growth of cases in
do_get_streams.
2019-02-28 22:32:05 -08:00
Eeshan Garg 85f453998e webhooks/zapier: Remove code related to our official Zapier app.
Our official Zapier app now uses our JS bindings (zulip-js) to
call the API directly and doesn't need to go through the webhook
anymore for anything.
2019-02-28 22:09:35 -08:00
Tim Abbott de65a04ae0 streams: Disable inline URL preview when rendering stream descriptions.
We want to use the baseline features of bugdown, but not fancy things
like inline URL previews, since the whole structure of stream
descriptions is to have a single-line thing supporting some
formatting.

The migration part of this change fixes a bug encountered by some
organizations upgrading from older versions of Zulip.
2019-02-28 17:00:40 -08:00
Tim Abbott d6c09eac51 bugdown: Add support for no_previews argument.
This allows us to have some features using bugdown rendering where
inline image previews will not be rendered (which would be problematic
for e.g. stream descriptions).
2019-02-28 16:54:04 -08:00
Tim Abbott 62dc6dda49 guests: Block guest user access to default streams.
Guest users will just get an empty list of default streams; we also
hide the "Default streams" organization view from the guest users UI.

This is for consistency with not providing guest users the full list
of streams in an organization.
2019-02-28 16:36:30 -08:00
Tim Abbott 1ce0e8256b zoom: Avoid sending Zoom API secret to other admin clients.
Fixing this involves fixing the backend to handle unchanged field
submissions of the Zoom credentials without trying to re-validate the
credentials (for performance) as well as to fetch the already-sent
secret.
2019-02-28 15:43:42 -08:00
Greg Price 9869153ae8 push notif: Send a batch of message IDs in one `remove` payload.
When a bunch of messages with active notifications are all read at
once -- e.g. by the user choosing to mark all messages, or all in a
stream, as read, or just scrolling quickly through a PM conversation
-- there can be a large batch of this information to convey.  Doing it
in a single GCM/FCM message is better for server congestion, and for
the device's battery.

The corresponding client-side logic is in zulip/zulip-mobile#3343 .

Existing clients today only understand one message ID at a time; so
accommodate them by sending individual GCM/FCM messages up to an
arbitrary threshold, with the rest only as a batch.

Also add an explicit test for this logic.  The existing tests
that happen to cause this function to run don't exercise the
last condition, so without a new test `--coverage` complains.
2019-02-26 16:41:54 -08:00
Challa Venkata Raghava Reddy e7fb19c8b0 invitations: Fix email validation errors for deactivated accounts.
This provides a much clearer error message when trying to invite a
user who has a deactivated account.

Fixes part of #8144.
2019-02-22 14:27:22 -08:00
Rohitt Vashishtha 57b9991396 markdown: Change syntax of silent mentions ( _@person -> @_person). 2019-02-20 10:41:42 -08:00
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
Tim Abbott 7a49611a94 settings_invites: Display links for multi-use invitations.
This makes it possible to figure out which is which if you have
several active at the same time.
2019-02-15 14:28:07 -08:00
Vishnu Ks 763eca6ca9 invites: Add UI for revoking multiuse invites. 2019-02-15 14:13:31 -08:00
Vishnu Ks 410e2574d1 settings: Remove unused setting ACCOUNT_ACTIVATION_DAYS.
ACCOUNT_ACTIVATION_DAYS doesn't seems to be used anywhere.
INVITATION_LINK_VALIDITY_DAYS seems to do it's job currently.

(It was only ever used in very early Zulip commits).
2019-02-15 13:56:10 -08:00
Eeshan Garg 179b747769 streams: Refactor multi-option helpers into separate functions.
For internal stream messages, most of the time, we have access to
a Stream object. For the few corner cases where we don't, it is a
much cleaner approach to have a separate function that accepts a
stream name than having one multi-option helper that accepts both
names and objects.
2019-02-12 11:10:26 -08:00
Eeshan Garg 3470e541c8 internal_send_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with re-fetching the Stream object by name.
2019-02-08 08:59:03 -08:00
Eeshan Garg c240008edb internal_prep_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with getting a Stream object back.
2019-02-08 08:53:16 -08:00