Commit Graph

40098 Commits

Author SHA1 Message Date
Steve Howell 40b0c36d21 minor: Update comment for guest subscription access.
As my comment indicates, I would prefer to handle
this explicitly by raising JsonableError in an
else statement here, but it's not a big deal.

This function can probably be simplified with a
bit of work, mostly on the testing side to make
sure we are covering all edge cases, but that
is out of the scope of my current PR.
2021-01-21 15:04:07 -08:00
Nikhil Maske 719217be8f markdown_help: Make spoiler-button work by changing its tag.
The "a" tag of spoiler-button present in Markdown formatting page
disrupts click handlers. To fix this, "a" tag is changed to "span" tag.
2021-01-21 14:32:13 -08:00
Mateusz Mandera fcc8debc3a users: Use realm.host in dummy user addresses without email visibility.
By moving the relevant logic from realm.get_bot_domain to
get_fake_email_domain we will make realm.host be used (if possible) for
dummy user addresses. That is, instead of user11@zulipchat.com, the
address will become user11@subdomain.zulipchat.com.
2021-01-21 13:04:38 -08:00
Mateusz Mandera 2283aa8a62 bots: Use realm.host for bot email domain if possible.
With the change in d70e1bcdb7,
bots get email like bot@zulip.com with EXTERNAL_HOST="zulip.com",
rather than bot@subdomain.zulip.com, which was the old format. That's
not desirable, so with this commit, realm.host will be used when
possible and only falling back to FAKE_EMAIL_DOMAIN if needed.
2021-01-21 13:04:38 -08:00
Mateusz Mandera b15dd9147d create_user: Remove redundant argument of get_display_email_address. 2021-01-21 13:04:38 -08:00
Steve Howell c693ae8982 event tests: Cover do_update_user_status better.
We often send only one field (away or status_text)
to be updated.

So we have to make our schema support optional
keys.

As a result of the more flexible schema, we no
longer need to exempt the node fixtures from
our schema checks.
2021-01-20 13:17:32 -08:00
Steve Howell 09ff0c400d tools: Rename tool to check-schemas. 2021-01-20 13:17:32 -08:00
Steve Howell 36b1794c1d user_status: Fix bug with resetting away status.
The fix is pretty simple here--if the client
doesn't send an away status, then don't change
it.

I improved the tests to cover this case.

Fixes #17071
2021-01-20 13:59:35 -05:00
Riken Shah 5da304d902 puppeteer_test: Replace textContent to value for the compose box.
This commit fixes the broken assertions for checking if compose box
is empty.

Fixes: #17088
2021-01-20 11:45:41 -05:00
YashRE42 b047c03997 puppeteer: Update language_setting_status check for new language data.
Commit f0f6138f01 deleted the
translation for "Saved. Please <a class='reload_link'>reload</a> for
the change to take effect." which caused puppeteer test 16-settings to
fail as it had hardcoded the translation for German. This commit
changes the expected text and hence fixes the failing puppeteer test.
2021-01-20 06:35:41 -05:00
Aman Agrawal 84a7e9f837 subscriptions_overlay: Show buttons in next line as block at <576px.
When we switch subscription overlay from two column to one
column overlay, we also set stream buttons to show in next line.

575px because it the breakpoint used by bootstrap 4 for small
screens.
2021-01-19 17:49:10 -08:00
Tim Abbott f0f6138f01 i18n: Update translation data from Transifex. 2021-01-19 17:45:13 -08:00
Mateusz Mandera a9242d6dfc retention: Eliminate redundant recipient JOIN from cross-realm query.
Since recipient_id (id of the PERSONAL Recipient of the user) was
denormalized into the UserProfile model, this query can be simplified by
getting rid of the zerver_recipient JOIN.
2021-01-18 21:40:37 -08:00
Mateusz Mandera e3be6db73a retention: Eliminate redundant userprofile JOIN from cross-realm query. 2021-01-18 21:40:37 -08:00
Harsh kumar 3cb07bc998
docs: Fix type in remote.md.
spelling errors in remote.md
2021-01-17 21:05:48 -08:00
Tim Abbott 5a02b33f2e digest: Add a large block comment on correctness. 2021-01-17 11:37:59 -08:00
Steve Howell 1040fb7219 email digests: Remove handle_digest_email shim.
The previous commit made it so we only call the
shim in tests, so now we completely remove it.
2021-01-17 11:28:30 -08:00
Steve Howell bfa0bdf3d6 email digests: Process users in chunks of 30.
This should make the queue empty more quickly,
because we do bulk queries to prevent database
hops.
2021-01-17 11:28:30 -08:00
Steve Howell e0b451730a email digests: Extract get_new_streams.
This makes us more efficient when handling
multiple users.  We don't have to keep
sending the same two queries to the database.

Note that as part of this we eliminated
a failure mode for the obscure population
of users from whom both `user.is_guest` and
`user.can_access_public_streams()` returns
False.  We know this would have only affected
Zephyr users (by looking at the code), and
we know we don't actually process Zephyr
users for email digests (or else we would
have raised exceptions in the old code).
2021-01-17 11:28:30 -08:00
Steve Howell 23de94504f email digests: Query streams for messages up front.
This should save us many hops to the database when
we process users in bulk.
2021-01-17 11:28:30 -08:00
Steve Howell 3662bf2dcb minor: Rename stream_map -> user_stream_map. 2021-01-17 11:28:30 -08:00
Steve Howell 11c93aced5 minor: Rename user_profile -> user and avoid shadowing. 2021-01-17 11:28:30 -08:00
Steve Howell f8bbb7fea9 email digests: Use select_related("realm").
We mostly need realm_id, but when we go to build
message lists, we need realm.uri.

We could probably be more aggresive about using
`only` here, but for now I am just trying to
reduce hops to the database.
2021-01-17 11:28:29 -08:00
Steve Howell bb56f0ec0e minor: Move get_stream_map to module level.
This is a pure code move.
2021-01-17 11:28:29 -08:00
Steve Howell 52e2d5a733 email digests: Avoid long_term_idle check.
We want to exclude users with recent subscription
activity from emails, regardless of whether
the long_term_idle flag is set.
2021-01-17 11:28:29 -08:00
Steve Howell 162b372b93 email digests: Do one query for recent streams.
This is another way to limit hops to the database
when we process users in bulk.
2021-01-17 11:28:29 -08:00
rht 16583fd5b4 docs: Say that certbot wildcard has been released. 2021-01-17 11:17:09 -08:00
Alex Vandiver c2526844e9 worker: Remove SignupWorker and friends.
ZULIP_FRIENDS_LIST_ID and MAILCHIMP_API_KEY are not currently used in
production.

This removes the unused 'signups' queue and worker.
2021-01-17 11:16:35 -08:00
Alex Vandiver 01658e39a9 sentry: Verify version is supported, first.
Raven SDK does not send a `title` field.
2021-01-17 11:15:40 -08:00
Alex Vandiver d688e18de2 errors: Remove references to "deployment", use "host".
The `deployment` key was only set in `do_report_error`, which is now
only used in one codepath (the queue worker).  The logging handlers on
staging call notify_server_error directly, which omits the
`deployment` key.

Remove the odd one-of key, and instead simply do dispatch in
`do_report_error`.
2021-01-17 11:08:12 -08:00
Aman Agrawal 9b23e540df message_edit: Stop stream_header_colorblock from overlapping others. 2021-01-17 11:07:32 -08:00
Steve Howell 2382fa7a19 refactor: Pass stream_ids to is_subscriber_subset.
After this change all peer_data functions consistently
use stream_id rather than some "sub" object whose
data type is complicated by all sort of fields that
don't really concern how we track subscribers.
2021-01-17 10:40:17 -08:00
Steve Howell 355f44ef13 refactor: Only pass stream_id for set_subscribers.
The goal here is to make all our peer_data functions
basically work in id space.  Passing a full `sub`
to these functions is a legacy of when subscriber
info was attached to a full stream "sub" object,
but we don't care about anything sub-related
(color, description, name, etc.) when we are
dealing with subscriptions.

When callers pass in stream_id, you can be more
confident in a quick skim of the code that we're
not mutating anything in the "sub".
2021-01-17 10:40:17 -08:00
Steve Howell 6cc880c858 refactor: Extract peer_data.js.
This de-clutters stream_data a bit.  Since our
peer data is our biggest performance concern,
I want to contain any optimizations to a fairly
well-focused module.

The name `peer_data` is a bit of a compromise,
since we already have `subs.js` and we use
`sub` as a variable name for stream records
throughout our code, but it's consistent with
our event nomenclature (peer/add, peer/remove)
and it's short while still being fairly easy
to find with grep.
2021-01-17 10:40:17 -08:00
Steve Howell 5c2f6321f2 node tests: Eliminate set_subscribers mock. 2021-01-17 10:40:17 -08:00
Steve Howell 58dcc70a35 refactor: Remove subscribers from stream_data subs.
This sets us up to use better system-wide data structures
for tracking subscribers.

Basically, instead of storing subscriber data on the
"sub" objects in stream_data.js, we instead have a
parallel data structure called stream_subscribers.

We also have stream_create, stream_edit, and friends
use helper functions rather than accessing
sub.subscribers directly.
2021-01-17 10:40:17 -08:00
Steve Howell a175ce65cf minor: Inline add_sub into create_sub_from_server_data.
We now use add_sub only in tests.

The line to defensively initialize subscribers does
not get copied from add_sub, since we know that
create_sub_from_server_data always initializes
subscribers via set_subscribers.
2021-01-17 10:40:17 -08:00
Mateusz Mandera 3623681d30 message_edit: Don't rely on .recipient_id change not affecting recipient.
The codepath for moving a topic changes the message.recipient_id to the
id of the new recipient, but later, in update_messages_for_topic_edit,
it uses message.recipient when querying for messages with the matching
topic in the *old* stream (because those are the other messages that
need to be moved). This is a bug which happens to work fine, because in
Django 2, if message.recipient gets fetched first and then
message.recipient_id is mutated, message.recipient will not be altered
and thus will retain the outdated, previously fetched value.

In Django 3 changing .recipient_id causes .recipient to be updated to
the new Recipient objects, which is the Recipient of the *new* stream.
That will cause the bug to manifest.

This is a bugfix preparing for the upgrade to Django 3.
2021-01-17 10:39:46 -08:00
Mateusz Mandera f76202dd59 django3: Save language preference in a cookie rather than the session.
Support for saving it in the session is dropped in django3, the cookie
is the mechanism that needs to be used. The relevant i18n code doesn't
have access to the response objects and thus needs to delegate setting
the cookie to LocaleMiddleware.

Fixes the LocaleMiddleware point of #16030.
2021-01-17 10:38:58 -08:00
Steve Howell 04b6108e71 minor: Require keywords for verify_action. 2021-01-17 12:31:04 -05:00
Steve Howell 3df507be73 refactor: Clean up args for fetch_initial_state_data.
We now require explicit keywords for all arguments
to fetch_initial_state_data except user_profile.

We provide reasonable defaults to keep the test
code concise.
2021-01-17 12:31:04 -05:00
Alex Vandiver 08d716c741 registration: Re-use the redirect_to_email_login_url helper.
In the case of reusing a registration link, reuse the
redirect_to_email_login_url helper.  This does have the side effect of
now showing a "you've already registered" note, which did not happen
previously, but that seems probably for the best, since the user did
just click a "register" link.
2021-01-13 11:28:32 -08:00
Alex Vandiver ad3d25103b registration: Pre-fill the email when redirecting to login.
ecfafc05c0 shifted to using a different paramter name to hint that
the user had previously signed up -- and in so doing also stopped
pre-filling the "email" box.  Also send along the email box, to save
users time.
2021-01-13 11:28:32 -08:00
Tushar912 c60f48c889 registration: Move "already in realm" check outside of validation.
Checking for `validate_email_not_already_in_realm` again (after the
form already did so), but only in the case that the form fails to
validate, means that we may be spending time pushing totally invalid
emails to the DB to check.  In the case of emails containing nulls,
this can even trigger a 500 error from PostgreSQL.

Stop calling `validate_email_not_already_in_realm` in the form
validation. The form is currently only used in two places -- in
`accounts_home` and in `maybe_send_to_registration`.  The latter is
only called if the address is known to not currently have an account,
so checking in there is unnecessary; and in the former case, we wish
different behaviour (the redirect) than just validation failure, which
is all the validator can do.

Fixes #17015.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2021-01-13 11:28:32 -08:00
Tushar912 410bb8ad89 imports: Add better checking for subdomains.
Add a `--allow-reserved-subdomain` flag which allows creation of
reserved keyword domains.  This also always enforces that the domain
is not in use, which was removed in 0258d7d.

Fixes #16924.
2021-01-12 17:54:01 -08:00
rht d514b799ad slack-import: Explain current implementation about threads. 2021-01-12 10:21:32 -08:00
marchantia f03e4b9f04 docs: Fix typo in emoji.md.
Change emoji-map.json to emoji_map.json.
2021-01-12 10:20:35 -08:00
sushant52 6f0e8a9888 auth: Handle the case of invalid subdomain at various points.
Fixes #16770.
2021-01-11 22:29:50 -08:00
Siddharth Asthana 6c888977a6 change_subdomain: Create a deactivated realm on updating subdomain.
When changing the subdomain of a realm, create a deactivated realm with
the old subdomain of the realm, and set its deactivated_redirect to the
new subdomain.
Doing this will help us to do the following:
- When a user visits the old subdomain of a realm, we can tell the user
that the realm has been moved.
- During the registration process, we can assure that the old subdomain
of the realm is not used to create a new realm.

If the subdomain is changed multiple times, the deactivated_redirect
fields of all the deactivated realms are updated to point to the new
uri.
2021-01-07 14:15:22 -08:00
Aman Agrawal 42dfd98607 stream_sidebar_row: Show full stream name on hover.
This will help users see the full stream name for truncated
stream names in the left sidebar more easily.
2021-01-04 18:19:48 -08:00