Commit Graph

40269 Commits

Author SHA1 Message Date
Alex Vandiver d688e18de2 errors: Remove references to "deployment", use "host".
The `deployment` key was only set in `do_report_error`, which is now
only used in one codepath (the queue worker).  The logging handlers on
staging call notify_server_error directly, which omits the
`deployment` key.

Remove the odd one-of key, and instead simply do dispatch in
`do_report_error`.
2021-01-17 11:08:12 -08:00
Aman Agrawal 9b23e540df message_edit: Stop stream_header_colorblock from overlapping others. 2021-01-17 11:07:32 -08:00
Steve Howell 2382fa7a19 refactor: Pass stream_ids to is_subscriber_subset.
After this change all peer_data functions consistently
use stream_id rather than some "sub" object whose
data type is complicated by all sort of fields that
don't really concern how we track subscribers.
2021-01-17 10:40:17 -08:00
Steve Howell 355f44ef13 refactor: Only pass stream_id for set_subscribers.
The goal here is to make all our peer_data functions
basically work in id space.  Passing a full `sub`
to these functions is a legacy of when subscriber
info was attached to a full stream "sub" object,
but we don't care about anything sub-related
(color, description, name, etc.) when we are
dealing with subscriptions.

When callers pass in stream_id, you can be more
confident in a quick skim of the code that we're
not mutating anything in the "sub".
2021-01-17 10:40:17 -08:00
Steve Howell 6cc880c858 refactor: Extract peer_data.js.
This de-clutters stream_data a bit.  Since our
peer data is our biggest performance concern,
I want to contain any optimizations to a fairly
well-focused module.

The name `peer_data` is a bit of a compromise,
since we already have `subs.js` and we use
`sub` as a variable name for stream records
throughout our code, but it's consistent with
our event nomenclature (peer/add, peer/remove)
and it's short while still being fairly easy
to find with grep.
2021-01-17 10:40:17 -08:00
Steve Howell 5c2f6321f2 node tests: Eliminate set_subscribers mock. 2021-01-17 10:40:17 -08:00
Steve Howell 58dcc70a35 refactor: Remove subscribers from stream_data subs.
This sets us up to use better system-wide data structures
for tracking subscribers.

Basically, instead of storing subscriber data on the
"sub" objects in stream_data.js, we instead have a
parallel data structure called stream_subscribers.

We also have stream_create, stream_edit, and friends
use helper functions rather than accessing
sub.subscribers directly.
2021-01-17 10:40:17 -08:00
Steve Howell a175ce65cf minor: Inline add_sub into create_sub_from_server_data.
We now use add_sub only in tests.

The line to defensively initialize subscribers does
not get copied from add_sub, since we know that
create_sub_from_server_data always initializes
subscribers via set_subscribers.
2021-01-17 10:40:17 -08:00
Mateusz Mandera 3623681d30 message_edit: Don't rely on .recipient_id change not affecting recipient.
The codepath for moving a topic changes the message.recipient_id to the
id of the new recipient, but later, in update_messages_for_topic_edit,
it uses message.recipient when querying for messages with the matching
topic in the *old* stream (because those are the other messages that
need to be moved). This is a bug which happens to work fine, because in
Django 2, if message.recipient gets fetched first and then
message.recipient_id is mutated, message.recipient will not be altered
and thus will retain the outdated, previously fetched value.

In Django 3 changing .recipient_id causes .recipient to be updated to
the new Recipient objects, which is the Recipient of the *new* stream.
That will cause the bug to manifest.

This is a bugfix preparing for the upgrade to Django 3.
2021-01-17 10:39:46 -08:00
Mateusz Mandera f76202dd59 django3: Save language preference in a cookie rather than the session.
Support for saving it in the session is dropped in django3, the cookie
is the mechanism that needs to be used. The relevant i18n code doesn't
have access to the response objects and thus needs to delegate setting
the cookie to LocaleMiddleware.

Fixes the LocaleMiddleware point of #16030.
2021-01-17 10:38:58 -08:00
Steve Howell 04b6108e71 minor: Require keywords for verify_action. 2021-01-17 12:31:04 -05:00
Steve Howell 3df507be73 refactor: Clean up args for fetch_initial_state_data.
We now require explicit keywords for all arguments
to fetch_initial_state_data except user_profile.

We provide reasonable defaults to keep the test
code concise.
2021-01-17 12:31:04 -05:00
Alex Vandiver 08d716c741 registration: Re-use the redirect_to_email_login_url helper.
In the case of reusing a registration link, reuse the
redirect_to_email_login_url helper.  This does have the side effect of
now showing a "you've already registered" note, which did not happen
previously, but that seems probably for the best, since the user did
just click a "register" link.
2021-01-13 11:28:32 -08:00
Alex Vandiver ad3d25103b registration: Pre-fill the email when redirecting to login.
ecfafc05c0 shifted to using a different paramter name to hint that
the user had previously signed up -- and in so doing also stopped
pre-filling the "email" box.  Also send along the email box, to save
users time.
2021-01-13 11:28:32 -08:00
Tushar912 c60f48c889 registration: Move "already in realm" check outside of validation.
Checking for `validate_email_not_already_in_realm` again (after the
form already did so), but only in the case that the form fails to
validate, means that we may be spending time pushing totally invalid
emails to the DB to check.  In the case of emails containing nulls,
this can even trigger a 500 error from PostgreSQL.

Stop calling `validate_email_not_already_in_realm` in the form
validation. The form is currently only used in two places -- in
`accounts_home` and in `maybe_send_to_registration`.  The latter is
only called if the address is known to not currently have an account,
so checking in there is unnecessary; and in the former case, we wish
different behaviour (the redirect) than just validation failure, which
is all the validator can do.

Fixes #17015.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2021-01-13 11:28:32 -08:00
Tushar912 410bb8ad89 imports: Add better checking for subdomains.
Add a `--allow-reserved-subdomain` flag which allows creation of
reserved keyword domains.  This also always enforces that the domain
is not in use, which was removed in 0258d7d.

Fixes #16924.
2021-01-12 17:54:01 -08:00
rht d514b799ad slack-import: Explain current implementation about threads. 2021-01-12 10:21:32 -08:00
marchantia f03e4b9f04 docs: Fix typo in emoji.md.
Change emoji-map.json to emoji_map.json.
2021-01-12 10:20:35 -08:00
sushant52 6f0e8a9888 auth: Handle the case of invalid subdomain at various points.
Fixes #16770.
2021-01-11 22:29:50 -08:00
Siddharth Asthana 6c888977a6 change_subdomain: Create a deactivated realm on updating subdomain.
When changing the subdomain of a realm, create a deactivated realm with
the old subdomain of the realm, and set its deactivated_redirect to the
new subdomain.
Doing this will help us to do the following:
- When a user visits the old subdomain of a realm, we can tell the user
that the realm has been moved.
- During the registration process, we can assure that the old subdomain
of the realm is not used to create a new realm.

If the subdomain is changed multiple times, the deactivated_redirect
fields of all the deactivated realms are updated to point to the new
uri.
2021-01-07 14:15:22 -08:00
Aman Agrawal 42dfd98607 stream_sidebar_row: Show full stream name on hover.
This will help users see the full stream name for truncated
stream names in the left sidebar more easily.
2021-01-04 18:19:48 -08:00
Aman Agrawal e566e985e4 topic_edit: Store edit history in all the message affected.
Instead of just storing the edit history in the message which
triggered the topic edit, we store the edit history in all
the messages that changed. This helps users track the edit history
of a message more reliably.
2021-01-04 18:18:05 -08:00
cozyrohan 16d1ab3d5f webhooks/github: Fix repeating description for edits and updates.
This change updates the GitHub Integration webhook
get_opened_or_update_pull_request_body method so that
the description is only printed if it actually changes.
If the update event is a result of some other
attribute update, such as an asignee change, then the
description is not included in the message sent to
the zulip stream.

Fixes #16345
2021-01-04 14:34:17 -08:00
Aman Agrawal 6f31a2d49f node_tests: Fix suddenly failing tests for unknown reasons. 2021-01-02 09:32:47 -05:00
Greg Price c7d329544b shared: Bump version to 0.0.4. 2020-12-30 15:02:13 -08:00
Greg Price 0de1d03dd4 typing_status: Mark recipient array as allowing number subtypes.
Concretely, we'll use this with a `UserId` type which is an
"opaque type alias" of `number` -- it's secretly implemented as
simply `number`, and it can be consumed by anything that wants a
`number` (in other words, it's a subtype of `number`), but the
fact that it secretly just is `number` is private to the module
that defines the type.

As far as the typing_status code is concerned, allowing this to
be a subtype of `number` just means that the code doesn't ever
try to inject new numbers of its own into the recipients arrays
that it passes around.
2020-12-30 14:55:24 -08:00
Greg Price 9a15301f4a typing_status: Mark recipient array as read-only for Flow.
This type means that code consuming this value promises not to
mutate it.  It's useful partly for the sake of simply controlling
mutation, so that arrays can be passed around without making
defensive copies; and partly because it makes the type covariant
in the elements, rather than invariant.

That is, if a function takes a plain Array<number | null>, then you
can't pass it an Array<number>, because it might add a `null` to it.
But if it takes $ReadOnlyArray<number | null>, then you can.
In general, Array<S> <: $ReadOnlyArray<S> <: $ReadOnlyArray<T>
for any S <: T, where `<:` means "is a subtype of".

Marking this type as read-only means we can pass in a read-only
array without adding a fixme (equivalent to a mypy type-ignore) to
locally disable the type-checker, nor a redundant defensive copy.
2020-12-30 14:55:24 -08:00
Anders Kaseorg 93101c953f alerts: Prevent icons from spilling out of alerts.
Fixes #16911.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-30 07:55:29 -08:00
Anders Kaseorg 2ba22d273a requirements: Delete unused pip2.in symlink.
It’s unused since commit a486872a8e
(#14278).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-30 07:54:19 -08:00
Aman Agrawal c685d36821 hipchat_import: Remove tool from codebase.
Remove functions and scripts used by HipChat import tool and
those which will no longer be required in future.
2020-12-23 08:28:49 -08:00
Aman Agrawal 62d721e859 docs: Remove HipChat migration guide.
As of Feb 15th 2019, Hipchat Cloud and Stride
have reached End Of Life and are no longer
supported by Atlassian. Since it is almost 2 years
now we can remove the migration guides.
2020-12-23 15:43:13 +05:30
Anders Kaseorg df9e40491d ui_init: Unnest an if statement.
This is equivalent because we cannot have delta < 0 && delta > 0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:44:46 -08:00
Vishnu KS 9fe39646fa analytics: Specify exact end_time in realm summary query.
Fetchings rows with end_time within the last 25 hours would result
in the realmcount queries returning two rows for each realm
if the analytics page was opened within an hour since the
count stats were updated.
2020-12-22 16:44:31 -08:00
Vishnu KS 9d5a1271d4 analytics: Make last_successful_fill handle FillState.STARTED case properly.
Subtracting an hour from end_time is correct only for CountStats with
hourly frequency. For daily frequency we should subtract a day instead.
2020-12-22 16:44:31 -08:00
Vishnu KS 235a347639 analytics: Move last_successful_fill to CountStat.
This is a prep commit. Currenty we only pass CountStat.property
to last_successful_fill function. But it needs access to
CountStat.time_increment as well. We can pass the entire CountStat
object to the function as a workaround. But making last_successful_fill
a property of CountStat seems to be much more cleaner.
2020-12-22 16:44:31 -08:00
Vishnu KS 189e9a2759 analytics: Create time_increment property in CountStat. 2020-12-22 16:44:31 -08:00
Anders Kaseorg 6c989550ba dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Anders Kaseorg d72f7d1d69 eslint: Fix unicorn/prefer-number-properties.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/v24.0.0/docs/rules/prefer-number-properties.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Anders Kaseorg dd8e14d3e6 eslint: Fix unicorn/prefer-date-now.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/v24.0.0/docs/rules/prefer-date-now.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Anders Kaseorg 2f80415756 eslint: Fix unicorn/no-lonely-if.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/v24.0.0/docs/rules/no-lonely-if.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Anders Kaseorg 6acbcec39d eslint: Fix unicorn/error-message.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/v24.0.0/docs/rules/error-message.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Anders Kaseorg 0c4239e387 js: Convert Object.assign({…}, …) to spread syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-21 15:37:36 -08:00
Mateusz Mandera 160cc5120a api: Require can_create_users permission to create users via API.
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues.  We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
2020-12-21 13:20:21 -08:00
Mateusz Mandera c9b6d8ddad models: Remove redundant Meta.permissions on Realm model.
This is dead code leftover from the old way of handling admin
permissions.
2020-12-21 13:15:40 -08:00
Mateusz Mandera d0dc04a093 models: Rename is_api_super_user to can_forge_sender, 2020-12-21 13:15:39 -08:00
sahil839 2fa33be683 actions: Refactor check_message to change return dataclass instead of Dict.
We change the return type of check_message to be dataclass instead of
Dict[str, Any]. This refactoring helps us to understand the context of the
data structure returned by check_message clearly which was not possible
when using Dict.

SendMessageRequest class is added in zerver/lib/message.py inspite of it
not being used in that file itself just to maintain consistency as other
TypedDicts and dataclasses are defined in that file and to avoid circular
dependency as SendMessageRequest is being used in lib/widget.py as well.

We also rename local variable to 'send_request' for accessing
SendMessageRequest objects.
2020-12-21 12:55:30 -08:00
Tim Abbott 908025bdad runtornado: Avoid providing a URL for Tornado on startup.
The {addr} part isn't directly useful, since connections to Tornado
are done on localhost anyway, and made the development environment
output a bit more confusing.

Also, use the same phrasing for restarts we use for Django.
2020-12-20 12:27:51 -08:00
Tim Abbott 4ee58f408b process_fts_updates: Make normal development startup silent.
We run this tool at DEBUG log level in production, so we will still
see the notice on startup there; this avoids a spammy line in the
development environment output..
2020-12-20 12:19:49 -08:00
Tim Abbott 1f036f9bde tornado: Reduce logging of event queue load/dump.
This logging is really only potentially interesting in a development
environment when the numbers are nonzero.

In production, it seems worth logging for consistency reasons.

Probably we'll eventually redo this block by change the log level, but
this is good enough to despam the development environment startup
output.
2020-12-20 12:14:39 -08:00
Tim Abbott eca67135d1 run-dev: Simplify output and colorize link users should click. 2020-12-20 12:11:16 -08:00