Commit Graph

13230 Commits

Author SHA1 Message Date
sahil839 911854d0bf models: Add assert statement for full_members policy in has_permission.
This commit adds an assert statement in the last block of
has_permission which checks whether the policy_value is
POLICY_FULL_MEMBERS_ONLY. This assert statement is added
for readability.
2021-03-30 09:06:20 -07:00
vagrant 92f4d206a6 documentation: Update upload_file python api documentation.
The current API documentation uses call_endpoint to upload a file;
since we've added a custom helper in python-zulip-api, we should
document that cleaner approach.
2021-03-30 08:52:07 -07:00
Alex Vandiver 134db206a9 outgoing_webhook: Set an X-Smokescreen-Role on requests.
This header is used by the Smokescreen outgoing proxy to provide
identification.
2021-03-29 18:24:44 -07:00
Alex Vandiver bd37dc0b83 outgoing_webhook: Set the user-agent on all requests. 2021-03-29 18:24:44 -07:00
Alex Vandiver cb3e6df8b9 outgoing_webhook: Add a requests session on the webhook.
The session object provides a common place to set headers on all
requests, no matter which implementation.

Because the `headers` attribute of Session is not a true static
attribute, but rather exposed via overriding `__getstate__`, `mock`'s
autospec cannot know about it, and thus throws an error; in tests that
mock the Session, we thus must explicitly set the `session.headers`.
2021-03-29 18:24:44 -07:00
Alex Vandiver be100154dd outgoing_webhook: Type do_rest_call with fewer Anys. 2021-03-29 18:24:44 -07:00
Alex Vandiver a280905a89 outgoing_webhook: Join build_bot_request and send_data_to_server.
The existing organization, of returning an opaque blob from
`build_bot_request`, which was later consumed by
`send_data_to_server`, is not particularly sensible; the steps become
oddly split between the OutgoingWebhookWorker, `do_rest_call`, and the
`OutgoingWebhookServiceInterface`.

Make the `OutgoingWebhookServiceInterface` in charge of building,
making, and returning the request in one method; another method
handles extracting content from a successful response.  `do_rest_call`
is responsible for calling both halves of this, and doing common error
handling.
2021-03-29 18:24:44 -07:00
Alex Vandiver be706ea7a1 outgoing_webhook: Replace a weird cast in tests with a mock. 2021-03-29 18:24:44 -07:00
Alex Vandiver e00126a688 outgoing_webhook: Remove warnings from test by giving a response code. 2021-03-29 18:24:44 -07:00
Alex Vandiver d88e6fe3fa outgoing_webhook: Use json= to automatically set content-type. 2021-03-29 18:24:44 -07:00
sahil839 fbf5ff26af tests: Fix stream creation part in comments of stream-policy tests.
The comments in stream-policy tests in test_message_send.py specifies
the restriction of creating streams based on stream_post_policy. But
this restriction was removed in 9aaa61963 and we now allow everyone to
create all type of streams. So this commit fixes the stream creation
parts in comments.
2021-03-29 17:37:34 -07:00
sahil839 f620110ca0 tests: Fix moderator comment in new-members stream-policy test. 2021-03-29 17:37:34 -07:00
sahil839 6aa45c8e20 openapi: Fix typo in stream_post_policy sections of openapi docs.
The STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS option of stream_post_policy
was explained as "Only new members can post" in the api docs. It should
instead be "Only full members can post" and this commit fixes it.
2021-03-29 17:37:34 -07:00
shanukun f8ef7d56b9 refactor: Make acting_user a mandatory kwarg for do_set_realm_message_editing. 2021-03-29 15:51:45 -07:00
shanukun 4dc62f962b refactor: Make acting_user a mandatory kwarg for do_set_realm_authentication_methods. 2021-03-29 15:51:45 -07:00
shanukun c95061e9b9 refactor: Make acting_user a mandatory kwarg for do_deactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 8f3ae715c0 refactor: Make acting_user a mandatory kwarg for do_reactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 3c3d805dd1 refactor: Make acting_user a mandatory kwarg for do_change_user_role 2021-03-29 15:51:45 -07:00
shanukun 459710a897 refactor: Make acting_user a mandatory kwarg for do_set_realm_property. 2021-03-29 15:51:45 -07:00
Suyash Vardhan Mathur 27ddb554fb openapi: Fix non-checking of /events and /register schema.
The /events and /register endpoints were excluded from schema validations,
because they were earlier not completely documented. However, they can
now be added for proper checking. Removed them from excluded endpoints list
and fixed the documentation for /register and /events after the checking.
Fixes #17796.
2021-03-28 22:05:44 -07:00
m-e-l-u-h-a-n 1b8a5a3344 markdown: Refactor backend logic for handling user mention.
Backend logic for handling user mention was cluttered
because it was handled at two stages first in
get_possible_mentions_info while fetching mention data
based on the messsage and then later in UserMentionPattern
which handles processing of text for mention.

Ideally UserMentionPattern should depend on
get_possible_mentions_info only for data but there was a
shared logic between these two that made it hard to debug
any possible bugs.

Updates in this commit make both of these functions
coherent in terms of logic and also add appropiate
comments to improve readability of these functions.

There was also a hidden bug that if a user A is
mentioned in with @**name|id** then @**invalid|id**
again mentioned A because of the way we handled mentions
earlier. It is solved as a result of this refactor and
appropiate test has been added for this.

This has been tested manually as well as by adding new
test to address missing case.
2021-03-28 16:52:48 -07:00
Anders Kaseorg d55dc6f8f1 requirements: Upgrade python-zulip-api from Git.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 16:31:03 -07:00
Tim Abbott 96c61a1a41 events: Fix broken custom_profile_fields events logic.
I noticed this because the test_events.py tests had the extremely
weird pattern of calling the actual change function, and then testing
the `notify` function's state changes (which should always be noops),
rather than actually testing the state change function.

Fixing the test made it clear that the actual logic in events.py
simply did not handle deleting custom_profile_field_value elements
from user objects when a custom_profile_field object was deleted.

So we fix that bit of logic as well.

It appears this bug was unique -- at least we don't have any other
notify_* functions being used directly in test_events.py, and the
handful of state_change_expected=False entries are all events for data
not present in page_params.
2021-03-26 16:28:33 -07:00
shanukun bc2d58ad4a custom_profile_fields: Remove op field for the event.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.

* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
2021-03-26 16:28:33 -07:00
tushar912 2cf51139cf custom profile fields: Rename "CHOICE" to "SELECT" in frontend.
This requires a small backend change to the label.
2021-03-26 11:49:11 -07:00
Anders Kaseorg 38ffd47b90 js: Convert static/js/page_params.js to ES6 module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 10:17:56 -07:00
Tim Abbott 459bf379c2 dev_panel: Use correct content helper for portico pages.
In 1a12e112d9, this page was converted
to use portico styling, but we intentionally left this page not using
the portico_content class since we didn't want the header/footer.

We still don't want the header/footer clutter, so instead, we achieve
that same goal using the isolated_page flag.
2021-03-26 09:40:40 -07:00
Tim Abbott f5b7e5d934 puppeteer_tests: Use POST for flush_caches call. 2021-03-25 12:58:36 -07:00
Riken Shah 99f8be6a12 puppeteer_tests: Reset test environment after each run.
When running some tests multiple times in the same call,
were failing because of the data duplication.

This commit resolves that issue by resetting the test
environment (i.e: Re-cloning test database and clearing
cache) after each run.

Fixes #17607.
2021-03-25 12:58:36 -07:00
Riken Shah 1d23d13ef5 urls: Add /flush_caches URL to remove all the server-side cache.
This endpoint is introduced for the puppeteer test, to
clear all the caches after each run.
2021-03-25 12:58:36 -07:00
Riken Shah b8691fb252 models: Add `clear_client_cache` function to clear `get_client_cache`.
This is a prep commit, this function will be used when we
want to reset the test environment of the puppeteer test.
2021-03-25 12:58:36 -07:00
Aman Agrawal 4d43a1baa9 default_stream_groups: Keep list of streams sorted by stream name.
Since the list of streams returned by a query which is not sorted
can vary, the tests which use it become flaky.
NormalActionsTest.test_default_stream_groups_events became
flaky due to this and hopefully sorting the streams should
fix it.
2021-03-25 14:44:26 +05:30
Adam Birds 25658153ca integration: Update Zendesk Documentation.
I have updated the docs for the Zendesk to integration to include
numbers to make it easily readable.

Fixes part of #17633.
2021-03-25 01:40:38 -07:00
Tim Abbott c3998a3d3b integrations: Make Hello World example less confusing. 2021-03-25 01:40:38 -07:00
Adam Birds b9549250f3 integrations: Update Hello World Docs.
I have the updated the documentation page for the hello world
integration to include numbers to bring it up to standard and make it
more readable.

Fixes part of #17633.
2021-03-25 01:36:05 -07:00
Adam Birds 1dc4dfef92 integrations: Update deskdotcom integration docs.
I have updated the documentation for the deskdotcom integration to
include numbers for ease of readability.

Fixes part of #17633.
2021-03-25 01:35:57 -07:00
m-e-l-u-h-a-n 5ae896758d logging: use assertEqual to assert logging output.
This commit migrates some tests in test_stripe.py and
test_embedded_bot_system.py to use assertEqual instead of
assertRegexpMatches for asserting logs, as suggested here:
https://github.com/zulip/zulip/pull/16818#discussion_r569888862
2021-03-25 01:10:03 -07:00
m-e-l-u-h-a-n ffd4abaa4e logging: migrate test_import_export to use assertLogs.
This commit  migrates some of the backend tests in test_import_export
to use assertLogs(), instead of mock.patch() as planned in #15331.

Logs for tests in this file are suppressed and are not asserted as
that made changes to import/export codebase more fragile. As we
already have checks for the actual functionalities, it made less
sense to assert those logs.
2021-03-25 01:10:03 -07:00
Suyash Vardhan Mathur d3a3c6898c api docs: Add documentation of deactivate-own-user endpoint.
Currently, there was no markdown page for deactivate-own-user API
endpoint. Created deactivate-own-user.md for the API page and
created a new owner client in test-api to reactivate the client
deactivated during testing.

Also changed endpoint name from deactivate-my-account to
deactivate-own-user, for better consistency with other endpoints.

Fixes #16163.
2021-03-25 01:05:44 -07:00
Mateusz Mandera 82d6d925e5 cache: Delete user_profile_by_email_cache_key.
This is no longer used in any important place,
get_user_profile_by_email is meant to be used only in manage.py shell
now and thus there's no point in this function being cached.
2021-03-25 00:47:42 -07:00
Mateusz Mandera a094b80fb1 test_subs: Remove redundant cache_delete calls.
These caches should be getting automatically invalidated upon
realm.save().
2021-03-25 00:47:42 -07:00
Mateusz Mandera f147c42f9d actions: Change caching of create_mirror_user_if_needed.
Emails are not unique, so we can only sensibly cache using keys formed
with both email and realm.

This requires adding a new cache key function for caching by delivery
email - user_profile_delivery_email_cache_key.
2021-03-25 00:47:42 -07:00
m-e-l-u-h-a-n 2699048208 markdown: Extend user mention syntax to support user_id for mentioning.
Extend our markdown system to support mentioning of users
by id also. Following these changes, it would be possible
to mention users with @**|user_id** and silently mention
using @_**|user_id**.

Main intention for extending the mention syntax is to make
it convenient for bots to mention a users using their ids. It
is to be noted that previous syntax are also supported.

Documentation tweaked by tabbott for better readability.

The changes were tested manually in development server, and also
by adding some new backend and frontend tests.

Fixes: #17487.
2021-03-25 00:44:56 -07:00
akshatdalton 9c6d8d9d81 markdown: Add support to shorten GitHub links.
We add support to shorten links and test their shortening in
well-organized, clean manner that makes it trivial to extend the
GitHub approach for GitLab and perhaps other services.

We only shorten basic types of GitHub links (issue, PR, commit) that
fit a set of simple common patterns; the default behaviour of Autolink
is kept for everything else.

Logic added in frontend and backend Markdown Processor is identical.
This makes easy to extend the logic for other services like GitLab.

Fixes #11895.
2021-03-25 00:39:44 -07:00
Tim Abbott d89c405074 custom_profile_fields: Revert incorrect early-converted hunk.
In b220d29fed, we incorrectly migrated a
field exposed to the frontend, which resulted in the frontend
puppeteer tests failing in CI.
2021-03-24 14:22:08 -07:00
Suyash Vardhan Mathur 9840803c00 openapi: Fix schema for unread_msgs in /register.
The schema for unread_msgs was missing additionalProperties: false
that was causing tests to pass even with undocumented parameters
which were validated as an additional property. Set
additionalProperties to false and added documentation for missing
count variable.

Fixes #17728.
2021-03-24 13:03:36 -07:00
Tim Abbott 7e77320953 custom profile fields: Rename s/choice/select/ in tests. 2021-03-24 12:57:55 -07:00
tushar912 98a6bdbd4c custom profile fields: Rename "SELECT" field validator.
Rename the "SELECT" field validator so that it can be reused
with the upcoming "SELECT_MULTIPLE" field.
2021-03-24 12:54:51 -07:00
tushar912 b220d29fed custom profile fields: Rename "CHOICE" to "SELECT" in backend.
Rename the "CHOICE" field to "SELECT" in backend. This is
done to improve readability as a prep for the upcoming
"SELECT_MULTIPLE" field.
2021-03-24 12:54:51 -07:00
sahil839 6d419565b7 streams: Show error according to create_stream_policy on failure.
This commit changes the list_to_streams function to raise error
according to create_stream_policy value when a user cannot create
streams instead of same error for all cases.
2021-03-24 12:33:58 -07:00
sahil839 13e412d553 tests: Modify invite_to_stream_policy tests to check all cases.
This commit modifies test_user_settings_for_subscribing_other_users
to check all the possible cases including the cases when a user
can successfully subscribe other users along with the already
tested failure cases. This commit also adds checks for guest users
which was not present before.
2021-03-24 12:33:58 -07:00
sahil839 b45d6ef0d8 tests: Use do_change_user_role and do_set_realm_property.
This commit replaces the code which directly changes user.role,
realm.create_stream_policy and realm.waiting_period_threshold
with do_change_user_role and do_set_realm_property functions
in test_can_create_streams. This makes the code similar to the
other tests.
2021-03-24 12:33:58 -07:00
sahil839 683b6f7d65 tests: Refactor test_can_create_streams and test_can_subscribe_other_users.
We refactor test_can_create_streams and test_can_subscribe_other_users
in test_subs.py. We want to follow a specific order in such tests
which is just set the policy value one by one and then checking
that the role in policy returns true and role just below that returns
false. This approach is explained in detail below.

Following hierarchy of roles is considered for these tests -
1. Realm admin
2. Full members
3. Members
4. Guests.

Then if the policy is set to admins only, we check that the having
role as admin returns true and the role just below that, i.e. full
member returns false. Similarly, if the policy is set to members
only, we check that a member should return true and role below it
which is guest should return false. We basically follow these as
we can assume that if a user with particular role cannot do the
required task, then user with role below in the hierarchy would
be not allowed to do the task too.

This commit refactors the above mentioned two tests to have above
explained workflow.
2021-03-24 12:33:58 -07:00
sahil839 bc72d2dbc4 tests: Remove unnecessary use of do_change_user_role in test_subs.
This commit removes the unnecessary do_change_user_role function
in test_can_subcribe_other_users. This was added in 1aebf3cab
which replaced the multiple functions like do_change_is_admin
and do_change_is_guest with do_change_user_role.

Previously two functions do_change_is_admin and do_change_is_guest
were used because there were two flags is_realm_admin and is_guest
which were used to determine the role of a user. But then we added
a single field role to UserProfile and removed the multiple flags
and thus also replaced the different functions with a single
do_change_user_role. With addition of a new field role, two
different do_change_* functions were not needed as we only have
a role field instead of different flags, but this was missed in
1aebf3cab and this commit fixes it.
2021-03-24 12:33:58 -07:00
Tim Abbott 11ff59aa5e test_home: Fix missing do_change_plan_type import. 2021-03-23 16:12:04 -07:00
Vishnu KS 7bf3d6f21f tests: Use do_change_plan_type in test_show_plans. 2021-03-23 15:51:50 -07:00
shanukun d68a2677d2 event_schema: Add checker for restart event.
Part of #17568.
2021-03-23 12:16:20 -07:00
shanukun cfe0fa3788 event_schema: Add schema check for realm/deactivated event.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.

With several block comments by tabbott explaining the logic behind our
behavior here.

Part of #17568.
2021-03-23 12:16:16 -07:00
Tim Abbott daa9bbba61 events: Add assertions that all ops are implemented.
We discovered recently that some ops for events were just not
implemented in events.py (specifically, realm/deactivated).

Since our goal is for events.py to be complete, we add this bit of
hardening to ensure that it stays that way.
2021-03-23 11:50:52 -07:00
m-e-l-u-h-a-n 830c4acedc markdown: Fix invalid mention bug for stream and stream topic mention.
Modifies `StreamPattern` and `StreamTopicPattern` to inherit
from InlineProcessor instead of Pattern. This change is done
because Pattern stopped checking for matching patterns as soon
as it found a match which was not a valid stream. Due to this
all the subsequent mention failed, even if they were valid.
This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

Due to above changes verbose_compile is no longer used for
precompiling STREAM_LINK_REGEX, STREAM_TOPIC_LINK_REGEX as
adds ^(.*?) and (.*?)$ which cause extra overhead of matching
pattern which is not required. With new InlineProcessor these
extra patterns at beggining and end are not required.
So, StreamPattern and StreamTopicPattern now define their own
__init__ method for precompiling the regex.

Fixes #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n dadbba0c25 markdown: Fix invalid mention bug for user group mention.
Modifies `UserGroupMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user group. Due to this all
the subsequent user group mention failed, even if they were
valid. This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n c8979a5100 markdown: Fix invalid mention bug for user mention.
Modifies `UserMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user. Due to this all the
subsequent user mention failed. This bug was only present in
backend renderring due to markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
shanukun a4cb264885 openapi: Remove add operation from op list of stream event.
This removes the `add` from op list of stream event, as we do not
actually generate the stream/add event in the API, and when a stream
is created we identify it using the `create` operation.

(This was likely just a mistake introduced as a result of the fact
that `create` does not fit the normal naming scheme; probably
long-term we should actually migrate this to "add", but more important
for now is to document what's accurate).

Part of #17568.
2021-03-22 23:44:56 -07:00
Vishnu KS 5e0030927b models: Make get_human_admin_users optionally not return realm owners. 2021-03-22 18:10:03 -07:00
Vishnu KS 7a351edb60 models: Make get_admin_users_and_bots optionally not return owners. 2021-03-22 18:10:03 -07:00
Alex Vandiver e637004695 sentry: Do not assume context lines exist in stacktrace.
`context_line` may not be set.
2021-03-22 12:16:10 -07:00
Tim Abbott f121e40848 message: Record whether unread_msgs data is truncated.
This is preparatory work for investigating reports of missing unread
messages.

It's a little surprising that not test failed after adding the code
without API documentation.

Co-Author-By: Tushar Upadhyay (tushar912).
2021-03-21 19:48:13 -07:00
Sourabh f7ac4bbc5f doc: Correct the location of `Make a Zap` button.
In Zulip's documentation of Zapier, it's mentioned that location of 
the "Make a Zap" button is in the upper right, but it's in the upper left.
2021-03-21 17:39:53 -07:00
Steve Howell 7c9a766c29 tests: Test send_restart_events. 2021-03-21 10:46:55 -04:00
Anders Kaseorg 6364e1b5f3 requirements: Upgrade talon fork to 1.4.8.
https://github.com/mailgun/talon/pull/200

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-18 17:10:18 -07:00
Abhijeet Prasad Bodas a30ca8490d refactor: Allow custom msg strs in send_message_moved_breadcrumbs.
This is a prep commit which modifies the
`send_message_moved_breadcrumbs` function to take
message strings as input.
This is done to reuse the function in other places
like the /digress command.
2021-03-18 16:13:00 -07:00
Adam Birds 7fd7a1917b integrations: Update Zabbix Documentation.
I have added a note about the fact these instructions are for Zabbix 5.2
and above and the workflow for other versions of Zabbix may be
different.
2021-03-18 15:00:10 -07:00
Tim Abbott 88f351cdee outgoing_webhook: Simplify error handling logic.
Structurally, exception, failure_message, and status_code are mutually
exclusive in how this function is called, and it's best for the
function's flow to represent that.
2021-03-18 14:41:34 -07:00
Siddharth Asthana ec28a7555c outgoing_webhook: modify outgoing_webhook's 407 error message.
The message from the bot which triggered the 407 error message notifies
the bot owner about the exceptions as well in the error message. This
commit handles it more gracefully and shows a generic message.
2021-03-18 14:39:26 -07:00
Siddharth Asthana 5ec0860a2f outgoing_webhook: Add bot name when a outgoing_webhook is triggered.
The messages from the bot which were triggered by the outgoing_webhooks
didn't have the bot name in them. This commit adds the bot name to it
and makes the corresponding changes in the tests.
2021-03-18 14:39:26 -07:00
Strifel 209c89be10 ldap: Add option to limit user access to certain realms.
This adds an option for restricting a ldap user
to only be allowed to login into certain realms.
This is done by configuring an attribute mapping of "org_membership"
to an ldap attribute that will contain the list of subdomains the ldap
user is allowed to access. This is analogous to how it's done in SAML.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2021-03-18 11:19:59 -07:00
Anders Kaseorg 23088b5d78 markdown: Fix some Any annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-17 18:41:46 -07:00
Anders Kaseorg 9864907985 mypy: Correct typing.re imports to typing.
Although typing.re exists in the standard library, mypy has never
recognized it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-17 18:41:46 -07:00
Gaurav Pandey 44ff6da107 email-integration: Notify user on failure to send message via email.
On replying to an email notifcation from a stream where the user
does not come under the stream_post_policy will subsequently result
in a failure. In such a case, the user does not receive feedback
regarding the failure.

Notify the user via notification bot if their email
message failed to send.
Fixes #16642.
2021-03-17 14:56:17 -07:00
Adam Birds 3649da27bf integrations: Update Yo Documentation.
I have updated the documentation for the Yp integration to include
number for ease of readability.

Fixes part of #17633.
2021-03-16 15:29:47 -07:00
Adam Birds 795e41f317 integrations: Update Pingdom Documentation.
I have updated the Pingdom documentation to include numbers to increase
ease of readability.

Fixes part of #17633.
2021-03-16 15:29:47 -07:00
Adam Birds 8337dce913 integrations: Update HomeAssistant Documentation.
I have updated the docs for the homeassistant integration to include
numbers to increase visibility.

Fixies part of #17633.
2021-03-16 15:29:47 -07:00
Alex Vandiver 738f457309 unminify: Fix lookup if source map does not exist in disk.
If the client has an old version of the code which is not present on
the server, don't throw a 500; instead, default to the same `unable to
look up in source map` message is used when the line numbers don't
line up.
2021-03-16 14:46:18 -07:00
Abhijeet Prasad Bodas a1e75edb8a linkifiers: Remove outdated comment in tests.
After 34e39248fc, we do support
generic GitHub URLs. This test and the comment above
it were added in 043baa2af8.
2021-03-15 11:19:59 -07:00
Abhijeet Prasad Bodas 9223dced3b refactor: Rename filter to linkifier in frontend code and docs.
This only leaves `page_params.realm_filters`, which
will be changed in further commits along with the
API change.
2021-03-15 11:19:59 -07:00
Adam Birds fa8914085e integrations: Update Zabbix Documentation.
I have updated the documentation for the Zabbix integration to give the
correct instructions for the latest version of Zabbix (5.2). The old
instructions are now obsolete.

I have also updated the message that is PMd to a user if the webhook
doesn't receive a complete payload to also align with the new
instructions.
2021-03-14 19:33:23 -07:00
Tim Abbott e42354c917 do_create_realm: Require passing kwargs by name. 2021-03-14 08:50:02 -07:00
Mateusz Mandera d91d3a05b9 tests: Use do_create_realm where possible.
Using do_create_realm should be preferred over manual creation where
possible, as it creates more realistic data.
2021-03-14 08:50:02 -07:00
Mateusz Mandera 20fd9b1dec tests: Eliminate use of get_user_profile_by_email in tests. 2021-03-11 20:30:57 -08:00
Mateusz Mandera 03e54232cf integrations: Use get_user function in dialogflow integration.
Using get_user_profile_by_email is invalid, as it omits the realm, and
also fetches via .delivery_email - our convention is that .email is
supposed to be used for user-facing purposes like this.
2021-03-11 20:30:57 -08:00
Mateusz Mandera 132ce525b1 test_realm: Fix test_do_deactivate_realm_clears_user_realm_cache.
self.example_user("hamlet") uses get_user_by_delivery_email, so it
doesn't actually cache anything. This should use a cached function, like
the test below: test_do_change_realm_subdomain_clears_user_realm_cache.
2021-03-11 20:30:57 -08:00
Mateusz Mandera faf314c8d3 test_retention: Delete redundant get_user_profile_by_email call.
This does absolutely nothing and must be in the code accidentally.
2021-03-11 20:30:57 -08:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
Anders Kaseorg 0a09c9dfd7 markdown: Re-enable typeshed stub for Python-Markdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-10 11:49:59 -08:00
Alex Vandiver 1926daa244 import/export: Fix black formatting. 2021-03-08 13:08:35 -08:00
Cyril Pletinckx ba7da6d5c0 import/export: Fix deprecated authentication method for Slack.
The query string parameter authentication method is now deprecated for
newly created Slack applications since the 24th of February[1].  This
causes Slack imports to fail, claiming that the token has none of the
required scopes.

Two methods can be used to solve this problem: either include the
authentication token in the header of an HTTP GET request, or include
it in the body of an HTTP POST request. The former is preferred, as
the code was already written to use HTTP GET requests.

Change the way the parameters are passed to the "requests.get" method
calls, to pass the token via the `Authorization` header.

[1] https://api.slack.com/changelog/2020-11-no-more-tokens-in-querystrings-for-newly-created-apps

Fixes: #17408.
2021-03-08 12:56:37 -08:00
sahil839 9a432b0c3b events: Remove name field from update subscription events.
This commit removes name field from update subscription
events, as it is not used by any of the clients, and use
stream_id in the events code instead.
2021-03-07 22:03:24 -08:00
Tim Abbott b15cb27fcc docs: Add a document explaining email/push notifications.
With various fixes by Mateusz Mandera.
2021-03-05 15:24:25 -08:00
Tim Abbott 28d437672f webhooks: Fix spelling of milliseconds. 2021-03-05 12:22:50 -08:00
Gaurav Pandey 3d7462a0e7 integration: Add jotform integration.
Fixes #16554
2021-03-04 11:40:58 -08:00
Rex Ferrer d4c0578560 refactor: Integrate POSTRequestMock into HostRequestMock.
Minimized code duplication by integrating POSTRequestMock into
HostRequestMock and then updating the required files with
HostRequestMock.

Fixes part of #1211.
2021-03-03 21:52:05 -08:00
Anders Kaseorg e128f92ee1 migrations: Swap out deprecated JSONField import in-place.
A deprecated import shouldn’t be used even in a migration, since the
migration will need to remain runnable in the future.  We never needed
a migration for this switch anyway; we just needed to edit the old
migration, since no actual state changes are involved.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:05 -08:00
shanukun eaa8862bc3 missedmessage_hook: Fix inaccurate docstring.
If the client is no longer on the Internet,
receiver_is_off_zulip returns False, not True.
2021-03-03 16:17:55 -08:00
sahil839 4ca21a6982 users: Give moderators same permissions as that of full members.
This commit updates the stream creation, subscribing others to
stream, wildcard mention settings and stream post policy to allow
realm moderators even if they are new and the respective setting
is set to allow full members only.
2021-03-02 17:19:31 -08:00
sahil839 b4fd15d516 models: Rename is_new_member to is_provisional_member.
This commit renames the is_new_member property in models.py
to is_provisional_member which will return true for any user
who is not a full member. We will add a condition in further
commit such that this returns 'False' for a moderator as we
will initially give all the rights to moderator that a full
member has.
2021-03-02 17:19:31 -08:00
Tim Abbott 99001f5040 api docs: Fix claim that subject won't appear for PMs.
This was inaccurate after testing the implementation, and there's an
argument that we shouldn't move it as it will simplify migrating to a
world where (some) private message threads can have topics.
2021-03-01 20:35:13 -08:00
Mateusz Mandera 6c7eee0a1f presence: Document that a user can disable their presence updates.
This is a crucial detail, as it makes UserPresence an unreliable
indicator of when users were recently active. It should be documented
more clearly.
2021-03-01 15:05:17 -08:00
sahil839 b53c773987 events: Remove email field from update subscription events.
This commit removes email field from update subscription
events, as email field is of no use in this case.
2021-03-01 14:52:06 -08:00
Tushar Upadhyay 6250902edc
webhooks: Add support for more GitHub pull_request events.
Add support for the locked, unlocked, auto-merge-enabled, and auto-merge-disabled actions 
for the pull_request event.

Fixes #16258.
2021-03-01 14:16:08 -08:00
Mateusz Mandera 6f9f608225 test_home: Fix wrong bot references in test_people.
These are all referring to email_gateway_bot, when they're supposed to
refer to the notification and welcome bots, respectively. The values are
the same though, so the tests were passing anyway.
2021-02-28 17:02:37 -08:00
Sumanth V Rao 829f9272d2 hotspots: Extract INTRO_HOTSPOTS from ALL_HOTSPOTS.
Its likely that we would implement new hotspots that aren't
a part of the tutorial hotspots, in the future. For instance,
a hotspot to advertise new features. Hence, grouping them into
categories like INTRO_HOTSPOTS would be a good start. We also
have an aggregate of all types of hotspots we may add in the
future, under ALL_HOTSPOTS.
2021-02-26 15:02:48 -08:00
Suyash Vardhan Mathur 82f6bff0c4 api docs: Sort response keys in /get-events.
Currently, the keys were not sorted in example responses of events
making them unreadable. Added sort_keys parameter to sort them.
2021-02-26 15:01:37 -08:00
Mateusz Mandera d91d1cba96 actions: Simplify the conditionals in revoke_preregistration_users.
This is a refactor to make the ifs easier to reason through.
2021-02-26 08:26:43 -08:00
Mateusz Mandera 22ac0f152e actions: Change prereg_user.status in revoke_preregistration_users.
It's clearer to have all the logic adjusting PreregistrationUser
statuses in one place rather than scattered.
2021-02-26 08:26:43 -08:00
Mateusz Mandera c651bed0d4 actions: Extract revoke_preregistration_users function. 2021-02-26 08:26:43 -08:00
Mateusz Mandera 4b903c5dcd invites: Fix bug revoking user invites in other realms than intended.
Fixes #17238.
In process_new_human user, the queries were wrong, revoking all invites
sent to the email address, even in other realms than the one where the
new account just got created.
2021-02-26 08:26:43 -08:00
Mateusz Mandera b9c1fed18c invites: Delete old compat code in the invites queue worker.
1.7.* is old enough at this point that we can clean up this code.
2021-02-26 08:26:43 -08:00
shanukun fafe1a31d7 refactor: Make acting_user a mandatory kwarg for do_activate_user. 2021-02-25 17:58:00 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Suyash Vardhan Mathur 0d01e7070f api docs: Add missing ID fields for event responses.
Some event responses were missing the id field. Added the ID at appropriate places.
2021-02-25 17:55:09 -08:00
Tim Abbott 101bccc867 api docs: Clarify when custom emoji events are sent. 2021-02-25 17:51:44 -08:00
neiljp (Neil Pilgrim) 3cad47bf4c api docs: Clarify realm_emoji update event returns all custom emoji. 2021-02-25 17:49:33 -08:00
Alex Vandiver e53be6d043 email: Set an envelope-from which may be different from the From: field.
The envelope-from is used by the MTA if the destination address is not
deliverable.  Route all such mail to the noreply address.
2021-02-24 17:32:28 -08:00
Suyash Vardhan Mathur f1eda37836 api docs: Fix documentation of update-subscription-settings.
The description of request parameter of update-subscription-settings was
wrongly pasted in yaml and wasn't completely removed from the md file.
Made appropriate fixes in yaml and md file.
2021-02-24 13:28:58 -08:00
Mateusz Mandera 1d4badf6ad tests: Test internal_send_private_message can send to cross-realm bots. 2021-02-23 15:26:47 -08:00
Mateusz Mandera 51d7f24d20 actions: Remove realm argument to internal_send_stream_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera 09fc79f911 actions: Remove realm argument to internal_send_private_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera a652573169 tests: Fix tests causing internal_send_private_message with wrong realm.
test_signup: This test was wrong, because the inviter UserProfile was
from a different realm. Such a PreregistrationUser shouldn't be
considered valid.

test_tutorial: The direct call to internal_send_private_message was
using sender's realm as the realm argument which is not valid. It
doesn't lead to any error because the codepath seems to mostly not care
about the realm arg if the sender is a cross-realm bot. From my reading
of the code I think that wrong realm arg here would break user mentions,
because it makes its way to check_message() and then to
build_message_send_dict - but overall the message gets sent without
errors. Either way, this was a bug in the test and should be fixed.
2021-02-23 15:26:47 -08:00
Suyash Vardhan Mathur dd8964a31f api docs: Fix id and type fields of events and display them.
Currently, the ID and Type fields didn't have a description,
and weren't being displayed. Added a schema component to add
descriptions, and display on the api page. Fixes part of #15967.
2021-02-23 15:22:53 -08:00
sahil839 d71afc5a26 actions: Include ROLE_MODERATOR in realm_user_count_by_role.
This commmit includes ROLE_MODERATOR in realm_user_count_by_role.

We also update test_change_role in test_audit_log.py to include
changes for moderator role as well.
2021-02-23 15:01:14 -08:00
sahil839 6b5cf231a1 users: Add new user 'shiva' as realm moderator.
Note that at this point, it's not possible to create moderator users;
this just will make it easier to write tests for logic involving them
as we develop the feature.
2021-02-23 15:00:49 -08:00
sahil839 af64ca388a models: Add realm moderator role.
Have not included "ROLE_MODERATOR" in UserProfile.ROLE_TYPES
in this commit because did not want to update the openapi
docs at this stage as it will be a user-facing change and
not updating the openapi docs with moderator role included in
UserProfile.ROLE_TYPES gives error in ./tools/check-schemas.
2021-02-23 15:00:09 -08:00
LoopThrough-i-j d1ee2d31c7 examples-python-api: Fix `message_id` type.
The `message_id` was made an `str` object because
the request expected `Dict[str, str]`. The request is now
casted to `Dict[str, Any]` to fix the issue and removed
typecast of `message_id` to str.

python-zulip-api reference:
https://github.com/zulip/python-zulip-api/pull/653
2021-02-23 08:58:00 -08:00
sahil839 15e74a637c tests: Check cases when full members and their bots can send messages.
Currently there are only tests for verifying the error case and there
are no tests to check the case where messages are sent successfully
in 'STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS' stream.

This commit adds tests for checking that full members and bots owned
by them can send message successfully in streams with post policy as
'STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS'.
2021-02-18 18:38:52 -08:00
sahil839 81ae29d461 stream: Allow new bot to send message if its owner is full member.
We currently not allow new bots to send message in stream with post
policy as 'STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS', but we should
allow them to send messages if their owner is a full member.

This will make it consistent with behavior in stream with post
policy as 'STREAM_POST_POLICY_ADMINS_ONLY' where we allow non admin
bots with owner as admin to send messages.
2021-02-18 18:38:52 -08:00
sahil839 3df87d0901 stream: Fix error handling in access_stream_for_send_message.
According to tests we should not allow bot without owners to
post in streams with STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS.
But the code does not handle this and the related test passes
and raises error for case of bots without owner because the bot
is itself a new member.

This commit fixes this by adding a condition to check if there
is no bot owner and then raise error if there is no owner.
2021-02-18 18:38:52 -08:00
Abhijeet Prasad Bodas fc0488fdb1 actions: Rename notify_topic_moved_streams function.
This is a minor refactor which renames the
notify_topic_moved_streams function to
send_message_moved_breadcrumbs.

This is done because this function will be also used
for other things in the future, when moving streams
or when using the /digress command, for example.
2021-02-16 17:28:59 -08:00
Suyash Vardhan Mathur 96bfeeb9e6 api docs: Expand checking for deprecated fields.
Added assertion to check that if a deprecated flag is in a field's
schema, then it should have deprecated mentioned in description
as well, and moved these checks to a separate function.
Fixes part of #15967.
2021-02-16 15:34:52 -08:00
Tim Abbott a3e1b9161c openapi: Document deprecation for max_message_id field. 2021-02-16 08:08:40 -08:00
Sumanth V Rao 540cca595c hotspots: Fix typos in function name and code comment. 2021-02-15 18:33:21 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Anders Kaseorg 1212083218 webhooks: Strengthen format_pull_request_event type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg d001676728 streams: Fix compose_views type safety.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg b728727d9d timeout: Remove unnecessary varargs support.
Mypy can check it this way.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg 77b7914cd7 test_helpers: Strengthen some decorator types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg a1ba3ca066 import_util: Strengthen get_users type using a Protocol.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg 3117969f03 webhooks: Remove get_event_handler ritual and Any casts.
It looks like this ritual was born when a type comment wasn’t working
because it was mistyped without the colon.

Signed-off-by: Anders Kaseorg <anders@zulip.com>'
2021-02-15 17:05:28 -08:00
Anders Kaseorg dd2a3b45cd test_service_bot_system: Strengthen for_all_bot_types decorator type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg 04a5e0c339 test_report: Avoid Any type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg 6eb1705068 cache: Strengthen ignore_unhashable_lru_cache decorator type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Shanu 7f196967ad event_queue: Remove internal fields being leaked to the API.
A few internal fields used for tracking which types of notifications
have already been sent for a given message, like `hander_id` and the
`push_notified` bundle of fields were being incorrectly included in
message events delivered to clients clients.

One could argue these fields might be useful hints to clients, but
because notifications can be triggered later on via
`missedmessage_hook`, they have no useful purpose in the API.

This commit move these extended event field on a `internal_data`
object within the event object, and delete this field in `contents()`
for call points that would serve data to clients.

Tweaked by tabbott to provide a cleaner interface.

We're not bumping API_FEATURE_LEVEL because these fields have always
been documented as being present only due to a bug, so no clients
should be expecting or relying on them.

Fixes: #15947.
2021-02-14 21:42:19 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 9773c0f1a8 python: Fix string literal concatenation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:02:51 -05:00
Mateusz Mandera 90636d5e81 events: Fix bug in get_recent_conversations_recipient_id.
user_profile.id was confused for user_profile.recipient_id. These bugs
are particularly sneaky as they can go undetected by tests due to ids of
objects accidentally coinciding. We add a mitigation for this class of
mistakes by shifting the Recipient.id sequence in test db.

This was introduced in dda3ff41e1.
On the rare occasion where user_profile.id would coincide with
recipient_id passed to the function, we would return the wrong value.
That is, instead of correctly returning recipient_id, we would return
sender.recipient_id - recipient id of the sender of the message, thus
possibly returning user_profile.recipient_id (if user_profile is the
sender) - exactly the situation the function wanted to avoid
with the `if recipient_id == my_recipient_id:` if. Ultimately resulting
in incorrect/malformed data in
state['raw_recent_private_conversations'].
2021-02-09 17:45:34 -08:00
Mateusz Mandera b8c8ea5262 tests: Fix bugs confusing recipient.type_id for other ids.
These tests were accidentally passing due to numbers coinciding.
2021-02-09 17:45:34 -08:00
Vishnu KS 3f4f16f4f1 digest: Remove comments from get_hot_topics.
The code is self explanatory.
2021-02-09 10:35:47 -08:00
Vishnu KS e9587900e6 digest: Use heapq.nlargest instead of sorted.
nlargest is the natural fit for selecting n biggest items
from an unsorted list. It's more readable as well as more
efficent (even though we don't care much about the efficeny
in this particular case).
2021-02-09 10:35:47 -08:00
Vishnu KS 738d759e6f digest: Create MAX_HOT_TOPICS_TO_BE_INCLUDED_IN_DIGEST constant. 2021-02-09 10:35:47 -08:00
Vishnu KS c0bd05b52d digest: Check whether length of hot topics is 4.
The length of hot topics would not exceed 4.
2021-02-09 10:35:47 -08:00
Vishnu KS 5c026d67e3 digest: Sort topics in descending order in get_hot_topics.
We want topics with high diversity and large lengths.
So they should be sorted with reverse=True.

This bug seems to be introduced in 936171d258
2021-02-09 10:35:47 -08:00
Suyash Vardhan Mathur c9c40d4fd2 api docs: Cleaned up CSS for parameter classes.
Deduplicated CSS classes of data types of response and
request parameters in API Documentation to use a single
class.
2021-02-09 10:31:36 -08:00
Suyash Vardhan Mathur 9d74c7001d api docs: Fix non-rendering response parameter data types.
The current logic doesn't display data types when the additionalProperties
variables are not object, but are array of strings, etc. Changed the if
condition to allow rendering in such cases.
2021-02-09 10:29:25 -08:00
Alex Vandiver d0f0c2f2ed digest: Fix the structure that we enqueue across when digesting.
This rename was missed in bfa0bdf3d6.
Without this fix, digest messages fail to send.
2021-02-08 17:28:59 -08:00
Steve Howell d0ba3cadcf minor: Clean up code formatting for do_create_user.
This makes the code easier to visually scan.
2021-02-08 09:07:04 -05:00
Anders Kaseorg d13a039b54 actions: Sort available_notification_sounds.
os.listdir uses an arbitrary filesystem-dependent order.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-07 06:33:55 -05:00
m-e-l-u-h-a-n 0e6343c071 users: Clarify readability issues related to access_user_by_id.
zerver/lib/users.py has a function named access_user_by_id, which is
used in /users views to fetch a user by it's id. Along with fetching
the user this function also does important validations regarding
checking of required permissions for fetching the target user.

In an attempt to solve the above problem this commit introduces
following changes:
1. Make all the parameters except user_profile, target_user_id
   to be keyword only.
2. Use for_admin parameter instead of read_only.
3. Adds a documentary note to the function describing the reason for
   changes along with recommended way to call this function in future.
4. Changes in views and tests to call this function in this changed
   format.

Changes were tested using ./tools/test-backend.

Fixes #17111.
2021-02-05 17:31:45 -08:00
Suyash Vardhan Mathur 26a81ab3aa api docs: Display data type of responses in API Documentation.
Previously, the data type of responses wasn't displayed in the API
Documentation, even though that OpenAPI data is carefully validated
against the implementation. Here we add a recursive function to
render the data types visibly in API Documentation.
Fixes part of #15967.
2021-02-05 10:41:42 -08:00
Suyash Vardhan Mathur 38dc1131b9 api docs: Minor fixes in documentation.
Added some missing response names and type in additionalProperties.
2021-02-05 10:41:42 -08:00
Suyash Vardhan Mathur 63c13a8f13 api docs: Fix documentation of update-subscription-settings.
The responses for the API weren't being rendered from yaml, and were
incorrectly formatted in yaml. The parameters also weren't completely
included in yaml and needed to be moved. Made appropriate fixes in
yaml and markdown file.
2021-02-05 10:41:42 -08:00
Anders Kaseorg ae0afa2390 markdown: Explode config dict.
Commit 434094e599 (#11321) changed this
from an Extension to a subclass of Markdown, so it no longer has any
reason to use a config dict structured like that of an Extension.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-05 10:52:31 -05:00
ritik 50b0496a3a bitbucket webhook: Fully deprecate username field. 2021-02-05 09:46:27 -05:00
m-e-l-u-h-a-n ccf520ff13 logging: Migrate many backend tests to use assertLogs.
This commit  migrates some of the backend tests to use assertLogs(),
instead of mock.patch() as planned in #15331.

Tweaked by tabbott to avoid tautological assertions.
2021-02-03 17:55:49 -08:00
m-e-l-u-h-a-n 7417ac9165 logging: Remove unncessary logging patches in backend tests.
There were some tests that had mock patches for logging, although no
logging was actually happening there. This commit removes such patches
in `corporate/tests/test_stripe.py`, `zerver/tests/test_cache.py`,
`zerver/tests/test_queue_worker.py`,
and `zerver/tests/test_signup.py`.
2021-02-03 17:47:38 -08:00
Vishnu KS edac24acf1 email_log: Inherit EmailLogBackEnd from smtp.EmailBackend.
EmailLogBackend used to create a new EmailMessage and copy
only certain values from the original EmailMultiAlternatives
object. This resulted in the loss of information and made
it harder to test PRs like
https://github.com/zulip/zulip/pull/17121.

So instead of creating a new EmailMessage, tweak and send the existing
EmailMultiAlternatives object.
2021-01-29 14:51:38 -08:00
Aman Agrawal b26727ed16 invite-new-users: Specify that the limit spans for the whole day. 2021-01-29 09:51:11 -08:00
Anders Kaseorg 69890f36b1 migrations: Enforce evaluation order in 0306 WHERE clause.
Depending on PostgreSQL’s query plan, it was possible for the value
condition to be evaluated before the field_type condition was checked,
leading to errors like

psycopg2.errors.InvalidDatetimeFormat: invalid value "stri" for "YYYY"
DETAIL:  Value must be an integer.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-29 09:35:52 -08:00
Ganesh Pawar a42f7a67e1 populate_db: Add images in test data.
This isn't quite the right model, because we're not actually going
through the upload code path, but it does at least provide some inline
image previews in the data.

Fixes part of #14991.
2021-01-27 17:52:28 -08:00
Anders Kaseorg 4ca66e7278 timezone: Correct common_timezones dictionary.
The changes are as follows:

• Fix one day offset in all western zones.
• Correct CST from -64800 to -21600 and CDT from -68400 to -18000.
• Disambiguate PST in favor of -28000 over +28000.
• Add GMT, UTC, WET, previously excluded for being at offset 0.
• Add ACDT, AEDT, AKST, MET, MSK, NST, NZDT, PKT, which the previous
  code did not find.
• Remove numbered abbreviations -12, …, +14, which are unnecessary.
• Remove MSD and PKST, which are no longer used.

Hardcode the dict and verify it with a test, so that future
discrepancies won’t go silently unnoticed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-27 15:23:15 -08:00
Anders Kaseorg a7bd1f8049 requirements: Upgrade Python requirements. 2021-01-26 13:27:50 -08:00
Anders Kaseorg c0ad595855 email_notifications: Fix HTML injection bug.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:50 -08:00
Anders Kaseorg 454144c35f queue_processors: Fix retry_send_email_failures type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:50 -08:00
Anders Kaseorg 70aa9903b9 list_realms: Convert percent formatting to "".format.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:22 -08:00
Anders Kaseorg c36a66cc1b redis_utils: Convert percent formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:22 -08:00
Mateusz Mandera 1432067959 dependencies: Upgrade to Django 3.1.
https://docs.djangoproject.com/en/3.1/releases/3.1/

- django.contrib.postgres.fields.JSONField is deprecated and should be
  replaced with models.JSONField
-  The internals of the implementation in the postgresql backend have
   changed a bit in
   f48f671223
   and thus we need to make an ugly tweak in test_runner.
- app_directories.Loader.get_dirs() now returns a list of PosixPath so
  we need to make a small tweak in TwoFactorLoader for that (PosixPath
  is not iterable)

Fixes #16010.
2021-01-26 10:20:00 -08:00
Mateusz Mandera bf9e5e52ce dependencies: Upgrade to Django 3.0.
Adjustments made due to changes in Django 3.0:
(https://docs.djangoproject.com/en/3.0/releases/3.0/)

- test_signup: INTERNAL_RESET_URL_TOKEN was moved to
  PasswordResetConfirmView.reset_url_token
- test_message_fetch:
  "add_never_cache_headers() and never_cache() now add the private
  directive to Cache-Control headers."
- "django.utils.html.escape() now uses html.escape() to escape HTML.
  This converts ' to &#x27; instead of the previous equivalent decimal
  code &#39;." - this requires adjusting the expected decimal code
  in some of the string fixtures in tests.
2021-01-26 10:20:00 -08:00
Aman Agrawal 961d1d0a68 community_topic_edit: Increase time limit to 3 days.
24hrs is a small time in an asynchronous conversation. Increased
time limit of topic editing for non-admins to 3 days.
2021-01-25 14:55:33 -08:00
Anders Kaseorg a873a68cc7 eslint: Fix unicorn/no-new-array.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-25 14:53:19 -08:00
akshatdalton 5f8a10124e url preview: Update Zulip User-Agent.
This commit updates the Zulip User-Agent to
'Mozilla/5.0 (compatible; ZulipURLPreview/{version}; +{external_host})'
as the older User-Agent was rendering Markdown YouTube titles as
'YouTube - YouTube'.

Fixes #16970.
2021-01-25 14:24:48 -08:00
Alex Vandiver 3381fad258 registration: Stop enqueueing to the signups queue.
c2526844e9 removed the `signups` queue
worker, and the command-line tool that enqueues to it -- but not the
automated process that enqueues during signups itself.

Remove the signup, since it is no longer in use.
2021-01-24 09:42:55 -08:00
Suyash Vardhan Mathur f4cf5166bb api docs: Display data type of parameters in API documentation.
Previously, the data type of parameters wasn't displayed in the API
Documentation, even though that OpenAPI data is carefully validated
against the implementation.  Here we add a recursive function to
render the data types visibly in the API documentation.

This only covers the request parameters; we'll want to do something
similar for response parameters in a follow-up PR.

Fixes part of #15967.
2021-01-21 15:56:07 -08:00
Steve Howell 1498b2ef69 apply_event: Fix broken deepcopy attempt for subs.
When we were getting an apply_event call for
a subscription/add event, we were trying not to
mutate the event itself, but this clumsy code
was still mutating the actual event:

    # Avoid letting 'subscribers' entries end up in the list
    for i, sub in enumerate(event['subscriptions']):
        event['subscriptions'][i] = \
            copy.deepcopy(event['subscriptions'][i])
        del event['subscriptions'][i]['subscribers']

This is only a theoretical bug.

The only person who receives a subscription/add
event is the current user.

And it wouldn't have affected the current user,
since the apply_event was correctly updating the
state, and we wouldn't actually deliver the event
to the client (because the whole point of apply_event
is to prevent us from having to piggyback the
super-recent events on to our payload or put
them into the event queue and possibly race).

The new code just cleanly makes a copy of each
sub, if necessary, as we add them to state["subscriptions"].

And I updated the event schemas to reflect that
subscribers is always present in subscription/add
event.

Long term we should probably avoid sending subscribers
on this event when the clients don't set something
like include_subscribers.  That's a fairly complicated
fix that involves passing in flags to ClientDescriptor.
Alternatively, we could just say that our policy is
that we never send subscribers there, but we instead
use peer_add events.  See issue #17089 for more
details.
2021-01-21 15:04:07 -08:00
Steve Howell c6acde9c63 apply_event: Use stream_ids, not names, for add/remove.
It's always cleaner to work in id space.  It probably
would have required a perfect storm to have broken
the existing code, but using ids is obviously more
robust in theory, and just as simple.
2021-01-21 15:04:07 -08:00
Steve Howell 0519f2d2b9 minor: Move include_subscribers guards in apply_event.
This sets us up for a cleaner diff in an
upcoming commit.
2021-01-21 15:04:07 -08:00
Steve Howell 3fa595ef85 minor: Clean up args for apply_event.
We now require keywords, so that there is no
pitfall for mixing up boolean parameters.
Positional parameters are basically evil
when you have a bunch of bools.

I also make user_profile the first argument.

Finally, the code is more diff-friendly.
2021-01-21 15:04:07 -08:00
Steve Howell e42baf9e13 minor: Clean up args for apply_events.
I eliminate the defaults, since the existing code
was already specificying values for most things.

I move all the booleans to the bottom for both
parameters and arguments.

I require explicit keywords for everything but
user_profile (which is now first).

And, finally, I format the code in a more
diff-friendly manner.
2021-01-21 15:04:07 -08:00
Steve Howell f2586d2f9b refactor: Introduce SubscriptionInfo dataclass.
We use this as the return type for
gather_subscriptions_helper and
get_web_public_subs, instead of tuples.
2021-01-21 15:04:07 -08:00
Steve Howell 768117f0ff refactor: Unify include_subscribers logic. 2021-01-21 15:04:07 -08:00