Commit Graph

13891 Commits

Author SHA1 Message Date
Eeshan Garg 0ac7d7d21d corporate: Store sponsorship request data in the database.
We are starting to run into situations where this data could be
quite useful for making future decisions, so it makes to store it
in the database, not just in an email.
2021-07-15 10:31:03 -07:00
Eeshan Garg 434c262365 analytics: Display org_type on /activity.
Moving forward we are hoping to collect data on org types from our
users, so it makes sense to display the org type on the "Counts"
tab of our /activity page.
2021-07-15 10:14:38 -07:00
Eeshan Garg faa695e86d registration: Collect organization type on sign-up. 2021-07-15 09:58:27 -07:00
Alex Vandiver 43f3f9221d slack_incoming: Handle explicit None text payload in webhook. 2021-07-15 09:37:39 -07:00
sahil839 2f44a08b37 settings: Convert settings_overlay.html to handlebars template.
Fixes #18796.
2021-07-15 08:59:23 -07:00
Tim Abbott 4fb6f089e5 api: Remove error for no settings changes requested.
While this error message was vaguely useful, it wasn't reachable via
the UI at all, and it prevents merging this endpoint with related
endpoints.
2021-07-14 15:54:05 -07:00
sahil839 e0faa9699d api: Remove incorrect realm setting in 'realm/update_dict' schema.
We incorrectly include many realm settings in the data section of
'realm/update_dict' schema. It should only contain the settings
related to message edit, realm icon, realm logo and authentication
methods and not other settings, becausea all the other settings send
'realm/update' event and not 'realm/update_dict' event.

This commit only removes 'add_emoji_by_admins_only' and others will
be removed separately.
2021-07-14 14:57:09 -07:00
sahil839 61acf68aa4 emoji: Allow emoji authors to delete the emoji always.
Previously, non-admin emoji authors were allowed to
delete the emoji only if add_emoji_by_admins_only
was false. But, as add_emoji_by_admins_only setting
is for who can add emoji and not delete emojis, it
should not affect the behavior of deleting emojis
and users should always be allowed to delete the
emojis which. they added themselves
2021-07-14 14:57:08 -07:00
sahil839 50240ca71b models: Use COMMON_POLICY_TYPES for user_group_edit_policy.
This commit adds moderators and full members options for
user_group_edit_policy by using COMMON_POLICY_TYPES.

Moderators do not require to be a member of user group in
order to edit or remove the user group if they are allowed
to do so according to user_group_edit_policy.
But full members need to be a member of user group to edit
or remove the user group.
2021-07-14 14:38:44 -07:00
sahil839 93a1479286 user_group: Change error message to "Insufficient permission".
There is no need to have a error message which specifies the
roles having permission to edit user-groups, we can simply
have error message as "Insufficient permission" as we already
show the roles having permission clearly in UI.
2021-07-14 14:38:43 -07:00
sahil839 2355e66850 models: Move UserProfile.notification_setting_types to UserBaseSettings.
Since all the notifications settings are defined in UserBaseSettings, we
should shift the notification_setting_types dict to UserBaseSettings.
2021-07-14 14:37:57 -07:00
sahil839 2f386aed0f models: Move UserProfile.property_types to UserBaseSettings.
Since all the display settings are defined in UserBaseSettings,
we should shift the property_types dict to UserBaseSettings.
2021-07-14 14:37:57 -07:00
sahil839 d7dfe80454 models: Add RealmUserDefault table for realm-level default of settings.
This table will be used to store the realm-level default of display
and notification settings for new users.
2021-07-14 14:35:04 -07:00
PIG208 c03b9c95ad request: Store client information using ZulipRequestNotes.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
2021-07-14 12:01:07 -07:00
PIG208 26875cdf0b tests: Migrate mocked Request to HostMockRequest.
This migrates some mocked Request class and mocked request achieved
with namedtuple in test_decorators and test_mirror_users to use the
refactored HostMockRequest.

Since weakref cannot be used with namedtuple, this old way of mocking a
request object should be migrated to using HostRequestMock. Only after
this change we can extract client from the request object and store it
via ZulipRequestNotes.
2021-07-14 12:01:07 -07:00
PIG208 8eb2c3ffdb request: Move realm from the request to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 5167a93229 request: Move tornado_handler to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 742c17399e request: Move miscellaneous attributes to ZulipRequestNotes.
This includes the migration of fields that require trivial changes
to be migrated to be stored with ZulipRequestNotes.

Specifically _requestor_for_logs, _set_language, _query, error_format,
placeholder_open_graph_description, saveed_response, which were all
previously set on the HttpRequest object at some point. This migration
allows them to be typed.
2021-07-14 12:01:07 -07:00
PIG208 5475334b16 request: Refactor to store requestor_for_logs in ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 3f9a5e1e17 request: Refactor to record rate limit data using ZulipRequestNotes.
We will no longer use the HttpRequest to store the rate limit data.
Using ZulipRequestNotes, we can access rate_limit and ratelimits_applied
with type hints support. We also save the process of initializing
ratelimits_applied by giving it a default value.
2021-07-14 12:01:07 -07:00
PIG208 da6e5ddcae request: Move log_data from HttpRequest to ZulipRequestNotes. 2021-07-14 12:01:05 -07:00
PIG208 75cde8ebcf request: Refactor HostRequestMock to subclass HttpRequest. 2021-07-14 11:52:43 -07:00
PIG208 03693cd27e request: Map HttpRequest to ZulipRequestNotes for typing.
We create a class called ZulipRequestNotes as a new home to all the
additional attributes that we add to the Django HttpRequest object.
This allows mypy to do the typecheck and also enforces type safety.

Most of the attributes are added in the middleware, and thus it is
generally safe to assert that they are not None in a code path that
goes through the middleware. The caller is obligated to do manual
the type check otherwise.

This also resolves some cyclic dependencies that zerver.lib.request
have with zerver.lib.rate_limiter and zerver.tornado.handlers.
2021-07-14 11:52:42 -07:00
akshatdalton f5c4d51ed2 resolve topic: Add `is:resolved` search keyword/filtering support.
This commit adds the backend support for `is:resolved` search keyword.
In the next commit, I will add the frontend support for the same.
2021-07-13 23:18:41 -07:00
akshatdalton 7ec406f39d refactor: Extract `RESOLVED_TOPIC_PREFIX` in topic.py.
This is a prep commit for #18990.
2021-07-13 23:18:41 -07:00
Abhijeet Prasad Bodas ebb4eab0f9 worker: Rewrite MissedMessageWorker to not be lossy.
Previously, we stored up to 2 minutes worth of email events in memory
before processing them. So, if the server were to go down we would lose
those events.

To fix this, we store the events in the database.

This is a prep change for allowing users to set custom grace period for
email notifications, since the bug noted above will aggravate with
longer grace periods.
2021-07-13 17:21:38 -07:00
Abhijeet Prasad Bodas 1709428cff models: Create MissedMessageEmailEntry table.
This will be used to store the missedmessage events received
during the waiting time for email notifications (which is currently
2 minutes, hardcoded).

The change in `test_retention` is because we've set `on_delete=CASCADE`
for the message field this table.
The new query is like so:
```
DELETE FROM "zerver_missedmessageemailentry"
    WHERE "zerver_missedmessageemailentry"."message_id" IN (
        1545, 1546, 1547, 1548, 1549, 1550, 1551, 1552, 1553
        )
```
2021-07-13 17:21:37 -07:00
Abhijeet Prasad Bodas c3319a5231 notification_data: Create common source for trigger strings.
This reduces loose strings in the codebase, and allows us to not worry
about the exact naming (`stream_email_enabled` or `stream_emails_enabled`?)
and tense (`mentioned` or `mention`?).

Ideally this new class should have been in `lib/notification_data.py`,
which is our file for things like this. But, the next commit requires
using this data in `models.py`, and importing from `notification_data.py`
to `models.py` causes recursive imports.
2021-07-13 17:16:32 -07:00
PIG208 e373df88e0 webhooks: Make github webhook support event filtering system. 2021-07-13 16:47:18 -07:00
Suyash Vardhan Mathur 309f4ba145 openapi: Make endpoint operationId dash-separated.
The operationId is directly used in URLs of API doc pages
to find the OpenAPI data to render. However, this is dash-
separated in the URLs, and having underscore_separated IDs
in OpenAPI data doesn't allow direct comparison of the two.

This commit changes all OperationIDs from underscore_separated
to dash-separated.
2021-07-13 16:46:22 -07:00
akshatdalton 3ea1ff7665 refactor: Use `assertGreaterEqual` helper instead of `assertTrue`. 2021-07-13 13:03:38 -07:00
akshatdalton 0b469e9e4e refactor: Use `assertGreater` helper instead of `assertTrue`. 2021-07-13 13:03:38 -07:00
akshatdalton e203112fd4 refactor: Use `assert_length` helper instead of `assertTrue/assertEqual`. 2021-07-13 13:03:38 -07:00
Alex Vandiver 0b7cff2f22 push_notifications: Hide "error" level messages from aioapns.
Work around Fatal1ty/aioapns#15, by silencing error-level logging from
the aioapns logger.  We deal with the results of failed
send_notification calls by examining the `result.description` and
handling them; the extra logging message merely clutters the Sentry
logs.
2021-07-13 09:28:05 -07:00
Suyash Vardhan Mathur 981e4f8946
openapi: Render all responses of an operation.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.

This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
2021-07-13 08:33:43 -07:00
Mateusz Mandera db7d2ee713 test_upload: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera af29bd77a0 test_sessions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera 91b3ba6d66 test_queue_worker: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera ccd961b797 test_reactions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera 6ec5a9698f test_markdown: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Abhijeet Prasad Bodas e63e86dcb2 worker: Ensure complete coverage for PushNotificationsWorker.
The `# nocoverage` was unnecessary apart from for the compatibility code,
so add a test for that code and remove the `# nocoverage`.

The `message_id` -> `message_ids` conversion was done in
9869153ae8.
2021-07-13 08:30:31 -07:00
aryanshridhar 7dc56fdcf2 settings_emoji: Allow only admins to override existing emojis.
Previously, even non-admins had the option to override built-in
emojis in the `Settings Emoji` UI.

This commits essentially limits the functionality of overriding
custom and allows only realm administrators to
override built-in emojis with their custom emojis by adding an
authorization check in the backend.

It also adds relevant tests in `test_realm_emoji` which tests
for the cases where an admin and non admin tries to override
the built-in emoji.

Fixes #18860.
2021-07-13 08:28:28 -07:00
Mateusz Mandera ab380b122b python_examples: Use ensure_users where appropriate.
We added this function in 8e1a7cfb52
in order to make things more readable in example which hard-code user
ids. The point is to validate that the id indeed refers to the user that
the person writing the example expects, while providing information to
readers of the code so they don't have to do db queries to figure out
the user. As mentioned in the commit referred to above, this is
particularly useful when some db changes cause renumbering of user ids -
because then all these ids have to be adjusted and it's nice to know the
intended user.
2021-07-13 08:24:06 -07:00
Anders Kaseorg 8486499314 fenced_code: Fix processor type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-09 17:10:31 -07:00
Tim Abbott eb8283c089 test_docs: Test the new case study page. 2021-07-09 13:19:05 -07:00
Mateusz Mandera c6bfd1aa88 management: Add change_password command.
Zulip identifies users by realm+delivery_email which means that the
Django changepassword command doesn't work well -
since it looks only at the .email field.
Thus we fork its code to our own change_password command.
2021-07-09 12:34:39 -07:00
Tim Abbott bc42ba87d4 message: Add types for RawUnreadMessagesResult keys.
This improves the readability for this important piece of code.
2021-07-09 11:34:07 -07:00
m-e-l-u-h-a-n 781179c56a subs: Rename subs.js to stream_settings_ui.js.
We use subs as a common variable name for a collection of stream
data structure used in settings, in lot of modules. So this
rename clears a bunch of related shadowed variables.
2021-07-09 09:38:58 -07:00
Tim Abbott c804ab27d5 actions: Rename do_activate_user to do_activate_mirror_dummy_user.
This function had a confusing name, which could result in someone
using it unintentionally when they meant do_reactivate_user.

We also add docstrings for both functions.
2021-07-08 17:33:17 -07:00
Mateusz Mandera 093ca7a574 zilencer: Log warning when a remote server hits rate limits.
We don't want this rate limit to affect legitimate users so it being hit
should be abnormal - thus worth logging so that we can spot if we're
rate limiting legitimate users and can know to increase the limit.
2021-07-08 15:55:03 -07:00