Commit Graph

12856 Commits

Author SHA1 Message Date
Mateusz Mandera 3623681d30 message_edit: Don't rely on .recipient_id change not affecting recipient.
The codepath for moving a topic changes the message.recipient_id to the
id of the new recipient, but later, in update_messages_for_topic_edit,
it uses message.recipient when querying for messages with the matching
topic in the *old* stream (because those are the other messages that
need to be moved). This is a bug which happens to work fine, because in
Django 2, if message.recipient gets fetched first and then
message.recipient_id is mutated, message.recipient will not be altered
and thus will retain the outdated, previously fetched value.

In Django 3 changing .recipient_id causes .recipient to be updated to
the new Recipient objects, which is the Recipient of the *new* stream.
That will cause the bug to manifest.

This is a bugfix preparing for the upgrade to Django 3.
2021-01-17 10:39:46 -08:00
Mateusz Mandera f76202dd59 django3: Save language preference in a cookie rather than the session.
Support for saving it in the session is dropped in django3, the cookie
is the mechanism that needs to be used. The relevant i18n code doesn't
have access to the response objects and thus needs to delegate setting
the cookie to LocaleMiddleware.

Fixes the LocaleMiddleware point of #16030.
2021-01-17 10:38:58 -08:00
Steve Howell 04b6108e71 minor: Require keywords for verify_action. 2021-01-17 12:31:04 -05:00
Steve Howell 3df507be73 refactor: Clean up args for fetch_initial_state_data.
We now require explicit keywords for all arguments
to fetch_initial_state_data except user_profile.

We provide reasonable defaults to keep the test
code concise.
2021-01-17 12:31:04 -05:00
Alex Vandiver 08d716c741 registration: Re-use the redirect_to_email_login_url helper.
In the case of reusing a registration link, reuse the
redirect_to_email_login_url helper.  This does have the side effect of
now showing a "you've already registered" note, which did not happen
previously, but that seems probably for the best, since the user did
just click a "register" link.
2021-01-13 11:28:32 -08:00
Alex Vandiver ad3d25103b registration: Pre-fill the email when redirecting to login.
ecfafc05c0 shifted to using a different paramter name to hint that
the user had previously signed up -- and in so doing also stopped
pre-filling the "email" box.  Also send along the email box, to save
users time.
2021-01-13 11:28:32 -08:00
Tushar912 c60f48c889 registration: Move "already in realm" check outside of validation.
Checking for `validate_email_not_already_in_realm` again (after the
form already did so), but only in the case that the form fails to
validate, means that we may be spending time pushing totally invalid
emails to the DB to check.  In the case of emails containing nulls,
this can even trigger a 500 error from PostgreSQL.

Stop calling `validate_email_not_already_in_realm` in the form
validation. The form is currently only used in two places -- in
`accounts_home` and in `maybe_send_to_registration`.  The latter is
only called if the address is known to not currently have an account,
so checking in there is unnecessary; and in the former case, we wish
different behaviour (the redirect) than just validation failure, which
is all the validator can do.

Fixes #17015.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2021-01-13 11:28:32 -08:00
Tushar912 410bb8ad89 imports: Add better checking for subdomains.
Add a `--allow-reserved-subdomain` flag which allows creation of
reserved keyword domains.  This also always enforces that the domain
is not in use, which was removed in 0258d7d.

Fixes #16924.
2021-01-12 17:54:01 -08:00
sushant52 6f0e8a9888 auth: Handle the case of invalid subdomain at various points.
Fixes #16770.
2021-01-11 22:29:50 -08:00
Siddharth Asthana 6c888977a6 change_subdomain: Create a deactivated realm on updating subdomain.
When changing the subdomain of a realm, create a deactivated realm with
the old subdomain of the realm, and set its deactivated_redirect to the
new subdomain.
Doing this will help us to do the following:
- When a user visits the old subdomain of a realm, we can tell the user
that the realm has been moved.
- During the registration process, we can assure that the old subdomain
of the realm is not used to create a new realm.

If the subdomain is changed multiple times, the deactivated_redirect
fields of all the deactivated realms are updated to point to the new
uri.
2021-01-07 14:15:22 -08:00
Aman Agrawal e566e985e4 topic_edit: Store edit history in all the message affected.
Instead of just storing the edit history in the message which
triggered the topic edit, we store the edit history in all
the messages that changed. This helps users track the edit history
of a message more reliably.
2021-01-04 18:18:05 -08:00
cozyrohan 16d1ab3d5f webhooks/github: Fix repeating description for edits and updates.
This change updates the GitHub Integration webhook
get_opened_or_update_pull_request_body method so that
the description is only printed if it actually changes.
If the update event is a result of some other
attribute update, such as an asignee change, then the
description is not included in the message sent to
the zulip stream.

Fixes #16345
2021-01-04 14:34:17 -08:00
Aman Agrawal c685d36821 hipchat_import: Remove tool from codebase.
Remove functions and scripts used by HipChat import tool and
those which will no longer be required in future.
2020-12-23 08:28:49 -08:00
Aman Agrawal 62d721e859 docs: Remove HipChat migration guide.
As of Feb 15th 2019, Hipchat Cloud and Stride
have reached End Of Life and are no longer
supported by Atlassian. Since it is almost 2 years
now we can remove the migration guides.
2020-12-23 15:43:13 +05:30
Vishnu KS 9fe39646fa analytics: Specify exact end_time in realm summary query.
Fetchings rows with end_time within the last 25 hours would result
in the realmcount queries returning two rows for each realm
if the analytics page was opened within an hour since the
count stats were updated.
2020-12-22 16:44:31 -08:00
Mateusz Mandera 160cc5120a api: Require can_create_users permission to create users via API.
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues.  We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
2020-12-21 13:20:21 -08:00
Mateusz Mandera c9b6d8ddad models: Remove redundant Meta.permissions on Realm model.
This is dead code leftover from the old way of handling admin
permissions.
2020-12-21 13:15:40 -08:00
Mateusz Mandera d0dc04a093 models: Rename is_api_super_user to can_forge_sender, 2020-12-21 13:15:39 -08:00
sahil839 2fa33be683 actions: Refactor check_message to change return dataclass instead of Dict.
We change the return type of check_message to be dataclass instead of
Dict[str, Any]. This refactoring helps us to understand the context of the
data structure returned by check_message clearly which was not possible
when using Dict.

SendMessageRequest class is added in zerver/lib/message.py inspite of it
not being used in that file itself just to maintain consistency as other
TypedDicts and dataclasses are defined in that file and to avoid circular
dependency as SendMessageRequest is being used in lib/widget.py as well.

We also rename local variable to 'send_request' for accessing
SendMessageRequest objects.
2020-12-21 12:55:30 -08:00
Tim Abbott 908025bdad runtornado: Avoid providing a URL for Tornado on startup.
The {addr} part isn't directly useful, since connections to Tornado
are done on localhost anyway, and made the development environment
output a bit more confusing.

Also, use the same phrasing for restarts we use for Django.
2020-12-20 12:27:51 -08:00
Tim Abbott 1f036f9bde tornado: Reduce logging of event queue load/dump.
This logging is really only potentially interesting in a development
environment when the numbers are nonzero.

In production, it seems worth logging for consistency reasons.

Probably we'll eventually redo this block by change the log level, but
this is good enough to despam the development environment startup
output.
2020-12-20 12:14:39 -08:00
Anders Kaseorg a054f57af6 message: Bundle message stripping, validation, and truncation.
We always want to do these at the same time.  Previously, message
editing did too much stripping (fixes #16837) and failed to check for
NUL bytes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-18 17:44:13 -08:00
sahil839 37c8505435 message: Raise exception when trying to mirror an already sent message.
Previously we were just returning a dict containing a message id when
trying to mirror a already sent message in 'zephyr_mirror' cases.

This commit changes this behaviour to raise an exception when trying
to mirror an already sent message by adding a new exception class
ZephyrMessageAlreadySentException and then the caller returns the
message_id directly, instead of calling do_send_messages which also
returns a list of size one containing the message_id only.

This is a prep commit for changing the return type of check_message to
be a dataclass instead of a Dict as now we have only single output for
check_message.
2020-12-18 16:40:11 -08:00
sahil839 4e99ec34a9 widget: Use different variable names for message and submessage content.
This commit renames the content variable in do_widget_post_save_actions
to message_content and is a prep commit for changing the return type of
check_message from Dict to dataclass.

This change is required because content variable is used two times in
this function - one for message content and other for submessage
content, so when we change the return type of check_message to
dataclass, the type of content variable is considered as str and then
when dict is assigned to content in the submessage case, mypy raises
'Incompatible types in assignment' error.

This issue is not faced before the dataclass migration because there is
no type checking for the values of dict returned by check_message as the
return type of check_message is 'Dict[str, Any]'.
2020-12-18 16:19:35 -08:00
sahil839 db85b8a236 actions: Change type of wildcard_mention_user_ids in message_dict to set.
The message_dict['wildcard_mention_user_ids'] should be empty set instead
of empty list when there are no wildcard mentions similar to the case
when there are wildcard mentions, where it is equal to set of user ids and
not list of user ids.
2020-12-18 16:17:26 -08:00
Anders Kaseorg 6b8f4782c4 test_mattermost_importer: Fix test for admins-to-owners change.
Commit ed498e2f8e forgot to update this
test.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-17 18:59:08 -08:00
Tim Abbott ed498e2f8e import: Import mattermost admins as Zulip owners.
Otherwise, we violate the invariant that all organizations have an owner.
2020-12-17 18:45:45 -08:00
Anders Kaseorg 2ab0b3d4fc validator: Reject ISO 8601 dates missing leading zeros.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 16:36:50 -08:00
Max Zawisa 0e40cc72af newrelic: Added owner field and cleaned up code.
I reformatted the tests and view to include information about who
acknowledged and closed the alert. Only includes the information about
the owner if there was an owner.

Made a few small changes to the refactored bit as requested in review.
2020-12-15 12:04:46 -08:00
Max Zawisa 57e847ab89 newrelic: refactor of time input handling.
Moved time formatting check and conversion to
zerver/lib/webhooks/common.py. Updated tests slightly to match new
output. Removed duration from the calculation because the difference
is less than the precision of output and it complicated the error
handling.
2020-12-15 12:04:46 -08:00
Max Zawisa ec00557962 docs: Updated New Relic documentation.
The docs are updated to work with the new webhook and new process on
https://one.newrelic.com.
2020-12-15 12:04:46 -08:00
Mateusz Mandera b652cc786c django3: Remove remaining postgresql_psycopg2 use.
Removed in Django 3.0.
2020-12-15 11:52:32 -08:00
angela s 64becb20b5
logging: Set decorator tests to use assertLogs.
Fixes part of #15331.
2020-12-15 11:46:25 -08:00
Alex Vandiver 438d2aa632 digests: Ensure that the teaser_data can be JSON-serialized.
Leaving this as a set means that it fails in zerver.lib.send_email
when serializing into a ScheduledEmail object.
2020-12-15 11:44:50 -08:00
Alex Vandiver 7c849fa940 slack: Check token access scopes before importing.
The Slack API always (even for failed requests) puts the access scopes
of the token passed in, into "X-OAuth-Scopes"[1], which can be used to
determine if any are missing -- and if so, which.

[1] https://api.slack.com/legacy/oauth-scopes#working-with-scopes
2020-12-15 11:33:15 -08:00
Anders Kaseorg 415897f491 api docs: Use normal async/await code in JavaScript examples.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 11:32:18 -08:00
Anders Kaseorg bf45f921a7 url_preview: Allow Beautiful Soup to get the charset from <meta>.
An HTML document sent without a charset in the Content-Type header
needs to be scanned for a charset in <meta> tags.  We need to pass
bytes instead of str to Beautiful Soup to allow it to do this.

Fixes #16843.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 11:30:57 -08:00
Siddharth Asthana daac7536f3 accounts/deactivated: Show deactivated_redirect url if present
If a user visits a realm which has been deactivated and it's
deactivated_redirect field is set, we should have a message telling the
user that the realm has moved to the deactivated_redirect url.
2020-12-14 21:04:52 -08:00
Siddharth Asthana 82f5759299 Realm: Add a deactivated_redirect URLField to Realm object.
We export a realm's data, and disable the realm, because the user
is moving from Zulip Cloud (e.g. https://example.zulipchat.com/) to
self-hosting or another platform (e.g. https://zulip.example.com/)
which we do not control. This commit adds a field in the realm object
called deactivated_redirect to store the url to which the realm has
moved.
2020-12-14 21:04:52 -08:00
Anders Kaseorg 2c5e9f65f8 eslint: Fix new-cap errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-10 19:52:22 -08:00
Puneeth Chaganti 5dc3489166 webhooks/sentry: Fix URL generated in transform_webhook_payload.
The URL incorrectly had `event` in the URL path, instead of `events`.

Closes #16783
2020-12-02 12:28:45 -08:00
Puneeth Chaganti b7a08323aa webhooks/sentry: Use received key when timestamp key is absent. 2020-12-02 12:28:45 -08:00
Sundar Guntnur cbb7fb8ac0 anchor_value: Fix parsing of large anchor values.
This handles the conditions when anchor values are larger than
LARGER_THAN_MAX_MESSAGE_ID by clamping them down to it.  Also added
tests for the function parse_anchor_value.

Fixes #16768.
2020-12-02 11:00:22 -08:00
Mateusz Mandera 43a0c60e96 exceptions: Make RateLimited into a subclass of JsonableError.
This simplifies the code, as it allows using the mechanism of converting
JsonableErrors into a response instead of having separate, but
ultimately similar, logic in RateLimitMiddleware.
We don't touch tests here because "rate limited" error responses are
already verified in test_external.py.
2020-12-01 13:40:56 -08:00
Steve Howell 92ce2d0e31 events: Fix apply_event for streams.
In 1bcb8d8ee8 I made
it so the webapp doesn't include "streams" in its
state from `fetch_initial_state_data`, but I didn't
address all the places in apply_event.
2020-12-01 13:01:38 -08:00
Steve Howell c566ecfb30 minor: Remove dead code in events test. 2020-12-01 13:01:38 -08:00
Vishnu KS dabbc3445a webhooks: Properly format the currency amount for refunds.
By default all Stripe API amounts are in the currency's smallest unit.
It's upto us to convert it to a bigger unit and show it to the end user.
And refund event used to show the currency in the smallest unit which makes
the output wrong when it comes to most currencies like USD, Europ, INR etc
which uses a bigger unit(eg Dollar instead of Cents) as the standard.
2020-11-29 18:11:24 -08:00
Max Zawisa f05a04e000
webhooks: Update NewRelic webhook for new format.
Update the New Relic webhook and tests to match the format specified
in the New Relic documentation. The new format sends a json body
instead of using url parameters. The old format is no longer supported
by New Relic according to their support staff; as a result, the fixtures for 
the old test cases were removed. Added fixtures for new test cases.

Fixes: #16393.
2020-11-18 16:19:08 -08:00
Anders Kaseorg 13e35bfa94 mypy: Use sqlalchemy-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:17:41 -08:00
Anders Kaseorg 8e0240300a message_fetch: Skip intermediate mutation in limit_query_to_range.
This avoids extra mypy annotations.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:17:41 -08:00