Commit Graph

1146 Commits

Author SHA1 Message Date
Ganesh Pawar fa928d5cd1 streams: Split setting for stream creation policy.
Users wanted a feature where they could specify
which users can create public streams and which users can
create private streams.

This splits stream creation code into two parts,
public and private stream creation.

Fixes #17009.
2021-10-01 10:26:42 -07:00
sahil839 9dd69c17ee settings: Add moderators and members option in delete_own_message_policy.
This commit adds moderators, full members and members options to
delete_own_message_policy in backend.
2021-09-30 14:59:31 -07:00
sahil839 909a3cde76 realm: Replace allow_message_deleting with delete_own_message_policy.
This commit replaces 'allow_message_deleting' boolean setting
with an integer setting 'delete_own_message_policy'. We have a
separate dropdown now for deciding which user-roles can delete
messages sent by themselves and the time-limit setting droddown
is different.

This new setting has two options - everyone and admins only. Other
options including moderators will be added further.

We also remove the "Never" option from the original time-limit
dropdown, as admins are always allowed to delete message. This
never option resembled the case of only admins being allowed to
delete but this state is now resembled by setting the dropdown
to "admins only" and we also disable the time-limit dropdown in
this case as admins are allowed to delete irrespective of limit.

Note, this setting is only for deleting messages sent by the
deleting user themselves, and only admins are allowed to delete
messages sent by others as before.
2021-09-30 14:59:31 -07:00
sahil839 b13bfa09c5 message: Make zero invalid value for message_content_delete_limit_seconds.
We make zero invalid value for message_content_delete_limit_seconds and
for handling the case of "Allow to delete message any time", the API-level
value of message_content_delete_limit_seconds is "anytime" and "None"
as the DB-level value. We also use these values for message retention
setting, so it helps maintain consistency.
2021-09-30 14:45:39 -07:00
Tim Abbott e556481ba0 streams: Remove duplicates of get_web_public_streams_queryset.
This is a somewhat subtle function, that deserves a few comments
explaining subtle details of its logic, and there's no good reason to
have multiple copies of that logic that are slightly inconsistent.

Because the main changes here are just checking for invariant
failures, the behavioral change here should be limited to ensuring
deactivated streams are not considered available even if they were
tagged as web public streams before deactivation.
2021-09-28 10:07:36 -07:00
Sahil Batra c233ee9935 settings: Migrate twenty_four_hour_time setting to RealmUserDefault.
This commit removes the existing default_twenty_four_hour_time field in
Realm table which was used to set the twenty_four_hour_time setting of
new user on joining and instead we now use the twenty_four_hour_time
field of RealmUserDefault table for the same.

With some tweaks by tabbott to clarify the documentation.
2021-09-23 10:44:42 -07:00
Mateusz Mandera 491f763f4e models: Improve conversion of CustomProfileField values.
These values are currently either a string already or a List[int]. We
should do the conversion in
do_update_user_custom_profile_data_if_changed properly: if the value is
already a string, it can be used directly - if it's not, orjson.dumps is
a more future-proof way of converting than str(). Using orjson.dumps
here also allows us to change the converter of the USER type
CustomProfileField to orjson.loads, which is nicer to have than
ast.literal_eval.
While orjson.dumps() and str() give the same output when
given the special case of List[int],
ast.literal_eval was previously used due to orjson.loads not being
a good inverse function to str in general. That gets straightened out
now.
2021-09-22 16:22:46 -07:00
Mateusz Mandera cacff28578 types: Fix declared type of custom profile field values.
None of the existing custom profile field types have the value as an
integer like declared in many places - nor is it a string like currently
decalred in types.py. The correct type is Union[str, List[int]]. Rather
than tracking this in so many places throughout the codebase, we add a
new ProfileDataElementValue type and insert it where appropriate.
2021-09-22 16:22:46 -07:00
Mateusz Mandera c4edda016f models: Remove property decorator from UserProfile.profile_data.
Having this as a property is misleading, given that it does db queries
to fetch the CustomProfileField data.
2021-09-22 16:22:46 -07:00
Tim Abbott 8fe61674d4 settings: Add web_public_streams_enabled server setting.
This new setting both serves as a guard to allow us to merge API
support for web public streams to main before we're ready for this
feature to be available on Zulip Cloud, and also long term will
protect self-hosted servers from accidentally enabling web-public
streams (which could be a scary possibility for the administrators of
a corporate Zulip server).
2021-09-21 10:50:53 -07:00
Abhijeet Prasad Bodas bc60109867 models: Add visibility policy indexes on UserTopic.
Part of #19272.
2021-09-17 12:14:28 -07:00
Abhijeet Prasad Bodas 2153aa9495 models: Add field UserTopic.visibility_policy.
Part of #19272
2021-09-17 12:14:28 -07:00
Abhijeet Prasad Bodas 4455dac424 models: Use default db table name for UserTopic.
Also update the realm import/export code and tests accordingly.
2021-09-17 12:14:28 -07:00
Abhijeet Prasad Bodas 2aea944a7e models: Rename UserTopic.date_muted to last_updated.
This is a follow-up to #19388.

We will in the future allow patch requests to change the visibility
of an existing topic, so `last_updated` is better name for this field.

This commit does not affect the API or events in any way, but only the
database.
2021-09-17 12:14:28 -07:00
Aman Agrawal 354491d037 models: Add is_web_public property to Realm model.
This will allow us easily know if a realm has web_public
enabled or not.
2021-09-14 10:33:53 -07:00
Riken Shah 8c31e6f96e emoji: Add backend changes to support still image for animated emojis.
Now, when we add a custom animated emoji to the realm
we also save a still image of it (1st frame of the gif). So
we can avoid showing an animated emoji every time.
2021-09-12 07:13:04 +00:00
shanukun 8c1ea78d7d invite: Extend invite api for handling expiration duration.
This extends the invite api endpoints to handle an extra
argument, expiration duration, which states the number of
days before the invitation link expires.

For prereg users, expiration info is attached to event
object to pass it to invite queue processor in order to
create and send confirmation link.
In case of multiuse invites, confirmation links are
created directly inside do_create_multiuse_invite_link(),

For filtering valid user invites, expiration info stored in
Confirmation object is used, which is accessed by a prereg
user using reverse generic relations.

Fixes #16359.
2021-09-10 16:53:03 -07:00
Sahil Batra 17087cf06f actions: Add do_set_realm_user_default_setting.
This commit adds do_set_realm_user_default_setting which
will be used to change the realm-level defaults of settings
for new users.

We also add a new event type "realm_user_settings_defaults"
for these settings and a "realm_user_settings_default" object
in '/register' response containing all the realm-level default
settings.
2021-09-09 10:38:38 -07:00
Tim Abbott 517c2ed39d models: Make realm.name no longer nullable.
This was likely initiall created with null=True in
5c5ffd6ea3 just because we didn't have a
plan for backfilling this field, but I verified that Zulip Cloud has
no realms without a name set, and that's the place most likely to have
any form of super-legacy nameless realms.

So we can clean up this aspect of the data model without a special
migration to do something with existing realms with name=None (which I
suspect would have resulted in a 500 anyway).
2021-09-08 17:38:56 -07:00
Anders Kaseorg 1376feda93 docs: Add missing space in “emoji set”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 15:31:28 -07:00
Wesley Aptekar-Cassels f743dda008 emoji: Change default emojiset from Google classic to Google modern.
This is a step towards updating our emoji to support newer Unicode
emojis (#19371).

See CZO for discussion:
https://chat.zulip.org/#narrow/stream/2-general/topic/emoji.20version.3F/near/1239756
2021-09-09 02:00:36 +05:30
Sahil Batra de4c0d9fe9 settings: Validate desktop_icon_display_count value in 'PATCH /settings'. 2021-09-08 11:04:44 -07:00
Shelly 6d92534d51 models: Add setters for is_realm_owner and is_moderator.
This fixes a regression where one could end up deactivating all owners
of a realm when trying to synchronize LDAP with the `is_realm_admin`
flag configured in `AUTH_LDAP_USER_FLAGS_BY_GROUP`.

With tweaks by tabbott to add is_moderator as well.

Fixes #18677.
2021-09-07 17:15:39 -07:00
Tim Abbott fecdc234bf models: Rename setting-changed RealmAuditLog entry.
We will likely in the future want to start creating these for all
settings changes, and this new name will make the code more readable
when we do so.
2021-09-07 10:49:56 -07:00
Dinesh 430c5cb8e7 user_settings: Create _legacy dicts for existing settings.
Since 84742a0, all settings are sent in the `user_settings` dictionary
which were previously sent inline with other fields in /register
response.

In order to simplify the process of adding new personal settings, we
want to transition to a world where new settings only need to consider
the `property_types` object, and code that needs to reference the
legacy behavior interacts with an object with `legacy` in its name.

This way, contributors working on new settings don't need to think
about the legacy code paths at all.

See https://chat.zulip.org/#narrow/stream/378-api-design/topic/user.20settings.20response.20in.20.2Fregister
to understand this better.
2021-09-07 10:14:45 -07:00
Sahil Batra 307b55e93d models: Move emojiset_choices method to UserBaseSettings class.
We move the emojiset_choices method from UserProfile class to
UserBaseSettings class because emojiset_choices exists in
UserBaseSettings class and this would be used for realm-level
settings as well along with existing user-level settings.
2021-09-01 13:29:32 -07:00
Alex Vandiver b011869777 migrations: Add a (profile,end) index on useractivityinterval.
The `user_activity_interval` worker calls:

```python3
    last = UserActivityInterval.objects.filter(user_profile=user_profile).order_by("-end")[0]
`````

Which results in a query like:
```sql
SELECT "zerver_useractivityinterval"."id", "zerver_useractivityinterval"."user_profile_id", "zerver_useractivityinterval"."start", "zerver_useractivityinterval"."end" FROM "zerver_useractivityinterval" WHERE "zerver_useractivityinterval"."user_profile_id" = 12345 ORDER BY "zerver_useractivityinterval"."end" DESC LIMIT 1
```

For users which have at least one matching row, this results in a
query plan like:
```
 Limit  (cost=0.56..711.38 rows=1 width=24) (actual time=0.078..0.078 rows=1 loops=1)
   ->  Index Scan Backward using zerver_useractivityinterval_7f021a14 on zerver_useractivityinterval  (cost=0.56..1031399.46 rows=1451 width=24) (actual time=0.077..0.078 rows=1 loops=1)
         Filter: (user_profile_id = 12345)
         Rows Removed by Filter: 98
 Planning Time: 0.059 ms
 Execution Time: 0.088 ms
```

But for users that have just been created, with no matching rows, this
is considerably more expensive:
```
 Limit  (cost=0.56..711.38 rows=1 width=24) (actual time=10798.146..10798.146 rows=0 loops=1)
   ->  Index Scan Backward using zerver_useractivityinterval_7f021a14 on zerver_useractivityinterval  (cost=0.56..1031399.46 rows=1451 width=24) (actual time=10798.145..10798.145 rows=0 loops=1)
         Filter: (user_profile_id = 12345)
         Rows Removed by Filter: (count of every single row in the table, redacted)
 Planning Time: 0.053 ms
 Execution Time: 10798.158 ms
```

Regular vacuuming can force the use of the index on `user_profile_id`
as long as there are few enough users, which is fast -- however, at
some point, the query planner decides that is insufficiently specific,
always chooses the effective-whole-table-scan.

Add an index on `(user_profile_id, end)`, which is expected to be
sufficiently specific that it is used even with large numbers of user
profiles.

Ref #19250.
2021-08-31 16:33:40 -07:00
Eeshan Garg 082dd99ed6 realm: Add datetime field for scheduled deletion of demo orgs.
This commit is a part of #19523.
2021-08-19 03:09:36 -07:00
Sahil Batra 4bd1dc0a56 models: Add is_system_group field to UserGroup model. 2021-08-19 02:54:29 -07:00
Anders Kaseorg 79f7f9b8d9 actions: Remove unused get_emails_from_user_ids function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Sahil Batra 5459a92e4a setting: Use "unlimited" instead of "forever" for retention setting.
This commit updates both the stream-level and realm-level message
retention setting to use 'unlimited' instead of 'forever' to set
message retention setting to "retain messages forever".
2021-08-08 15:56:57 -07:00
Mateusz Mandera f1d5f3904d cache: Flush realm caches after both pre_delete and post_delete signals.
Fixes #19468.
2021-08-04 12:13:22 -07:00
Hemanth V. Alluri c00089ac28 drafts: Add support for toggling drafts synchronization.
With changes mostly to the API documentation by tabbott.
2021-07-30 10:00:27 -07:00
Abhijeet Prasad Bodas de04f0ad67 notifications: Calculate PMs/mentions settings like other settings.
Previously, we checked for the `enable_offline_email_notifications` and
`enable_offline_push_notifications` settings (which determine whether the
user will receive notifications for PMs and mentions) just before sending
notifications. This has a few problem:

1. We do not have access to all the user settings in the notification
handlers (`handle_missedmessage_emails` and `handle_push_notifications`),
and therefore, we cannot correctly determine whether the notification should
be sent. Checks like the following which existed previously, will, for
example, incorrectly not send notifications even when stream email
notifications are enabled-
```
if not receives_offline_email_notifications(user_profile):
    return
```
With this commit, we simply do not enqueue notifications if the "offline"
settings are disabled, which fixes that bug.

Additionally, this also fixes a bug with the "online push notifications"
feature, which was, if someone were to:
* turn off notifications for PMs and mentions (`enable_offline_push_notifications`)
* turn on stream push notifications (`enable_stream_push_notifications`)
* turn on "online push" (`enable_online_push_notifications`)

then, they would still receive notifications for PMs when online.
This isn't how the "online push enabled" feature is supposed to work;
it should only act as a wrapper around the other notification settings.

The buggy code was this in `handle_push_notifications`:
```
if not (
    receives_offline_push_notifications(user_profile)
    or receives_online_push_notifications(user_profile)
):
    return

    // send notifications
```

This commit removes that code, and extends our `notification_data.py` logic
to cover this case, along with tests.

2. The name for these settings is slightly misleading. They essentially
talk about "what to send notifications for" (PMs and mentions), and not
"when to send notifications" (offline). This commit improves this condition
by restricting the use of this term only to the database field, and using
clearer names everywhere else. This distinction will be important to have
non-confusing code when we implement multiple options for notifications
in the future as dropdown (never/when offline/when offline or online, etc).

3. We should ideally re-check all notification settings just before the
notifications are sent. This is especially important for email notifications,
which may be sent after a long time after the message was sent. We will
in the future add code to thoroughly re-check settings before sending
notifications in a clean manner, but temporarily not re-checking isn't
a terrible scenario either.
2021-07-28 13:55:25 -07:00
Abhijeet Prasad Bodas 683c8507e4 models: Remove MutedTopic alias for UserTopic.
Part of #19272
2021-07-28 10:25:54 -07:00
Abhijeet Prasad Bodas 798defc046 models: Rename MutedTopic to UserTopic.
Part of #19272

We still keep refering to this model with "MutedTopic" to reduce the
diff size of this commit. The alias will be removed in the next commit.

This commit skips on renaming the `date_muted` field to something more
general. That will be done in further commits, along with the code and
API changes.
2021-07-28 10:25:53 -07:00
sahil839 1ce09d9db6 realm: Extend add_custom_emoji_policy to use COMMON_POLICY_TYPES.
This commit adds full members and moderators option for
add_custom_emoji_policy by using COMMON_POLICY_TYPES.
2021-07-27 16:41:23 -07:00
sahil839 5efc854f66 models: Add can_add_custom_emoji helper.
This commit adds can_add_custom_emoji
helper to check whether the user can
add custom emoji or not.

This function will be used further when
add_custom_emoji_policy will be extended
to include all COMMON_POLICY_VALUES.
2021-07-27 16:41:23 -07:00
sahil839 4b1313a92b models: Replace add_emoji_by_admins_only with add_custom_emoji_policy.
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
2021-07-27 16:41:22 -07:00
Sahil Batra 355f6e9b53 settings: Move 'enter_sends' setting to property_types dict.
This commit moves "enter_sends" setting to property_types dict.
With this change, changing enter_sends setting also sends an
event of type "update_display_settings" and thus enables us
to live-update the UI.
2021-07-27 15:52:53 -07:00
Mateusz Mandera 7af789c48f models: Make realm_id arg of get_system_bot required. 2021-07-27 15:51:59 -07:00
sahil839 8ced74192a invite: Add 'Nobody' option to invite_to_realm_policy in backend. 2021-07-26 17:48:01 -07:00
Mateusz Mandera 8f588dcbab models: Pass realm to get_user_including_cross_realm calls. 2021-07-26 15:33:13 -07:00
Mateusz Mandera d45f3eecaa models: Add optional realm_id argument to get_system_bot. 2021-07-26 15:31:10 -07:00
Tim Abbott 12737fdc02 models: Add models for emoji in user statuses. 2021-07-23 13:11:21 -07:00
Hemanth V. Alluri 08e9e48205 models: Add a new field controlling drafts synchronization.
This field will control whether or not a user wants to sync their
drafts between different clients.  Defaults to enabled.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2021-07-23 12:31:00 -07:00
Abhijeet Prasad Bodas d5d3f14eee models: Add UserProfile field for custom email notifs batching periods.
The default is 2 minutes (120 seconds), which was the hardcoded value
earlier.
2021-07-23 12:13:46 -07:00
Tim Abbott d7ef0c7232 billing: Fix options and help text for sponsorship requests.
This makes several changes:

* Fixes a bug where the help text explaining our policies was not displayed.
* No help text was defined for many organization types.
* Copy-edits the help text somewhat.
* Offers all of the organization type options.
* Removes the 100% coverage requirement because it's annoying to test
  the e.currentTarget click handler.
2021-07-18 20:48:27 -07:00
Anders Kaseorg 1ae56e466b cache: Fix typing for post_save and post_delete flush handlers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 13:14:04 -07:00