Commit Graph

40067 Commits

Author SHA1 Message Date
Akshat ffed6e3dc9
message view: Add hotkey hints.
'>' and 'e' are added as hotkey hints for 'Quote and
reply or forward' and 'View source / Edit topic' options
in actions popover, to help make these hotkeys more discoverable.
2021-01-25 14:26:43 -08:00
akshatdalton 5f8a10124e url preview: Update Zulip User-Agent.
This commit updates the Zulip User-Agent to
'Mozilla/5.0 (compatible; ZulipURLPreview/{version}; +{external_host})'
as the older User-Agent was rendering Markdown YouTube titles as
'YouTube - YouTube'.

Fixes #16970.
2021-01-25 14:24:48 -08:00
Alex Vandiver 3381fad258 registration: Stop enqueueing to the signups queue.
c2526844e9 removed the `signups` queue
worker, and the command-line tool that enqueues to it -- but not the
automated process that enqueues during signups itself.

Remove the signup, since it is no longer in use.
2021-01-24 09:42:55 -08:00
Ganesh Pawar eefa687832 run-dev: Suppress the notices made by third-party tools.
This limits the `run-dev.py` startup output.
And made the terminal message a bit more clear about
accessing the server.

Fixes #16846
2021-01-22 18:00:30 -08:00
Anders Kaseorg 50121cce5e push-to-pull-request: Add --merge option.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-22 18:00:25 -08:00
Anders Kaseorg 26a7b7196c push-to-pull-request: Use getopt.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-22 18:00:25 -08:00
Tim Abbott 63539ddc37 website: Mention that we sponsor hundreds of projects. 2021-01-22 10:02:26 -08:00
Tim Abbott 1636b893dc docs: Add additional clarifications around realm terminology. 2021-01-21 16:31:03 -08:00
Suyash Vardhan Mathur f4cf5166bb api docs: Display data type of parameters in API documentation.
Previously, the data type of parameters wasn't displayed in the API
Documentation, even though that OpenAPI data is carefully validated
against the implementation.  Here we add a recursive function to
render the data types visibly in the API documentation.

This only covers the request parameters; we'll want to do something
similar for response parameters in a follow-up PR.

Fixes part of #15967.
2021-01-21 15:56:07 -08:00
ganpa b1f4c98730 custom_emojis: Fix unresponsiveness after submitting without a name.
While adding custom emojis, when a user clicks on the submit
button without providing a name to the emoji, the submit button
becomes unresponsive. This commit fixes that.

Fixes #16921
2021-01-21 15:51:09 -08:00
Steve Howell 1498b2ef69 apply_event: Fix broken deepcopy attempt for subs.
When we were getting an apply_event call for
a subscription/add event, we were trying not to
mutate the event itself, but this clumsy code
was still mutating the actual event:

    # Avoid letting 'subscribers' entries end up in the list
    for i, sub in enumerate(event['subscriptions']):
        event['subscriptions'][i] = \
            copy.deepcopy(event['subscriptions'][i])
        del event['subscriptions'][i]['subscribers']

This is only a theoretical bug.

The only person who receives a subscription/add
event is the current user.

And it wouldn't have affected the current user,
since the apply_event was correctly updating the
state, and we wouldn't actually deliver the event
to the client (because the whole point of apply_event
is to prevent us from having to piggyback the
super-recent events on to our payload or put
them into the event queue and possibly race).

The new code just cleanly makes a copy of each
sub, if necessary, as we add them to state["subscriptions"].

And I updated the event schemas to reflect that
subscribers is always present in subscription/add
event.

Long term we should probably avoid sending subscribers
on this event when the clients don't set something
like include_subscribers.  That's a fairly complicated
fix that involves passing in flags to ClientDescriptor.
Alternatively, we could just say that our policy is
that we never send subscribers there, but we instead
use peer_add events.  See issue #17089 for more
details.
2021-01-21 15:04:07 -08:00
Steve Howell c6acde9c63 apply_event: Use stream_ids, not names, for add/remove.
It's always cleaner to work in id space.  It probably
would have required a perfect storm to have broken
the existing code, but using ids is obviously more
robust in theory, and just as simple.
2021-01-21 15:04:07 -08:00
Steve Howell 0519f2d2b9 minor: Move include_subscribers guards in apply_event.
This sets us up for a cleaner diff in an
upcoming commit.
2021-01-21 15:04:07 -08:00
Steve Howell 3fa595ef85 minor: Clean up args for apply_event.
We now require keywords, so that there is no
pitfall for mixing up boolean parameters.
Positional parameters are basically evil
when you have a bunch of bools.

I also make user_profile the first argument.

Finally, the code is more diff-friendly.
2021-01-21 15:04:07 -08:00
Steve Howell e42baf9e13 minor: Clean up args for apply_events.
I eliminate the defaults, since the existing code
was already specificying values for most things.

I move all the booleans to the bottom for both
parameters and arguments.

I require explicit keywords for everything but
user_profile (which is now first).

And, finally, I format the code in a more
diff-friendly manner.
2021-01-21 15:04:07 -08:00
Steve Howell f2586d2f9b refactor: Introduce SubscriptionInfo dataclass.
We use this as the return type for
gather_subscriptions_helper and
get_web_public_subs, instead of tuples.
2021-01-21 15:04:07 -08:00
Steve Howell 768117f0ff refactor: Unify include_subscribers logic. 2021-01-21 15:04:07 -08:00
Steve Howell e735ce3f01 refactor: Move subscribers logic up to caller.
The gather_subscriptions_helper function now updates
subscribers instead of delegating.
2021-01-21 15:04:07 -08:00
Steve Howell d9740045a5 refactor: Eliminate checks in build_stream_dict_for_sub.
We eliminate some redundant checks.

We also consistently provide a `subscribers` field
in our stream data with `[]`, even if our users
can't access subscribers.  We therefore bump
the API version and tweak the docs.  (See further
down for a detailed justification of the change.)

Even though it is sometimes fine to have redundant code
that is defensive in nature, some upcoming changes are gonna
move subscriber-related logic out of build_stream_dict_for_sub
for certain codepaths as part of our effort to streamline
the payload for subscribers within page_params.

So we can't rely on the code that I removed here
inside of build_stream_dict_for_sub.

Anyway, it makes more sense to do these checks explicitly
in the validate function.

The code in build_stream_dict_for_sub was almost effectively
a noop, since the validation function was already preventing
us from getting subscriber info.  The only difference it
made was sometimes converting `[]` to `None`, and then
subsequently omitting the subscribers field.

Neither ZT nor the webapp make any distinction between
`[]` or <missing key> for the `subscribers` data in
`page_params`.

The webapp has had this code for a long time (and now
equivalent code elsewhere in this PR):

    if (!Object.prototype.hasOwnProperty.call(sub, "subscribers")) {
        sub.subscribers = new LazySet([]);
    }

The webapp calculates access based on booleans, anyway:

    sub.can_access_subscribers =
        page_params.is_admin || sub.subscribed ||
        (!page_params.is_guest && !sub.invite_only);

And ZT would choke if `subscribers` were missing, except that
it never gets to the relevant code due to other checks:

    def get_other_subscribers_in_stream(<snip>):
        assert stream_id is not None or stream_name is not None

        if stream_id:
            assert self.is_user_subscribed_to_stream(stream_id)

            return [sub
                    for sub in self.stream_dict[stream_id]['subscribers']
                    if sub != self.user_id]
        else:
            return [sub
                    for _, stream in self.stream_dict.items()
                    for sub in stream['subscribers']
                    if stream['name'] == stream_name
                    if sub != self.user_id]

You could make a semantic argument that we should prefer
<missing key> to `[]` when subscribers aren't even available, but
we have precedent from the way that `bulk_get_subscriber_user_ids`
has traditionally populated its result:

    result: Dict[int, List[int]] =
        {stream["id"]: [] for stream in stream_dicts}

If we changed `stream_dicts` to `target_stream_dicts` we
would faciliate a move toward `None`, but it would just cause
headaches for other server code as well as the frontends
(which, to reiterate, already prefer the empty array
for convenience).
2021-01-21 15:04:07 -08:00
Steve Howell 40b0c36d21 minor: Update comment for guest subscription access.
As my comment indicates, I would prefer to handle
this explicitly by raising JsonableError in an
else statement here, but it's not a big deal.

This function can probably be simplified with a
bit of work, mostly on the testing side to make
sure we are covering all edge cases, but that
is out of the scope of my current PR.
2021-01-21 15:04:07 -08:00
Nikhil Maske 719217be8f markdown_help: Make spoiler-button work by changing its tag.
The "a" tag of spoiler-button present in Markdown formatting page
disrupts click handlers. To fix this, "a" tag is changed to "span" tag.
2021-01-21 14:32:13 -08:00
Mateusz Mandera fcc8debc3a users: Use realm.host in dummy user addresses without email visibility.
By moving the relevant logic from realm.get_bot_domain to
get_fake_email_domain we will make realm.host be used (if possible) for
dummy user addresses. That is, instead of user11@zulipchat.com, the
address will become user11@subdomain.zulipchat.com.
2021-01-21 13:04:38 -08:00
Mateusz Mandera 2283aa8a62 bots: Use realm.host for bot email domain if possible.
With the change in d70e1bcdb7,
bots get email like bot@zulip.com with EXTERNAL_HOST="zulip.com",
rather than bot@subdomain.zulip.com, which was the old format. That's
not desirable, so with this commit, realm.host will be used when
possible and only falling back to FAKE_EMAIL_DOMAIN if needed.
2021-01-21 13:04:38 -08:00
Mateusz Mandera b15dd9147d create_user: Remove redundant argument of get_display_email_address. 2021-01-21 13:04:38 -08:00
Steve Howell c693ae8982 event tests: Cover do_update_user_status better.
We often send only one field (away or status_text)
to be updated.

So we have to make our schema support optional
keys.

As a result of the more flexible schema, we no
longer need to exempt the node fixtures from
our schema checks.
2021-01-20 13:17:32 -08:00
Steve Howell 09ff0c400d tools: Rename tool to check-schemas. 2021-01-20 13:17:32 -08:00
Steve Howell 36b1794c1d user_status: Fix bug with resetting away status.
The fix is pretty simple here--if the client
doesn't send an away status, then don't change
it.

I improved the tests to cover this case.

Fixes #17071
2021-01-20 13:59:35 -05:00
Riken Shah 5da304d902 puppeteer_test: Replace textContent to value for the compose box.
This commit fixes the broken assertions for checking if compose box
is empty.

Fixes: #17088
2021-01-20 11:45:41 -05:00
YashRE42 b047c03997 puppeteer: Update language_setting_status check for new language data.
Commit f0f6138f01 deleted the
translation for "Saved. Please <a class='reload_link'>reload</a> for
the change to take effect." which caused puppeteer test 16-settings to
fail as it had hardcoded the translation for German. This commit
changes the expected text and hence fixes the failing puppeteer test.
2021-01-20 06:35:41 -05:00
Aman Agrawal 84a7e9f837 subscriptions_overlay: Show buttons in next line as block at <576px.
When we switch subscription overlay from two column to one
column overlay, we also set stream buttons to show in next line.

575px because it the breakpoint used by bootstrap 4 for small
screens.
2021-01-19 17:49:10 -08:00
Tim Abbott f0f6138f01 i18n: Update translation data from Transifex. 2021-01-19 17:45:13 -08:00
Mateusz Mandera a9242d6dfc retention: Eliminate redundant recipient JOIN from cross-realm query.
Since recipient_id (id of the PERSONAL Recipient of the user) was
denormalized into the UserProfile model, this query can be simplified by
getting rid of the zerver_recipient JOIN.
2021-01-18 21:40:37 -08:00
Mateusz Mandera e3be6db73a retention: Eliminate redundant userprofile JOIN from cross-realm query. 2021-01-18 21:40:37 -08:00
Harsh kumar 3cb07bc998
docs: Fix type in remote.md.
spelling errors in remote.md
2021-01-17 21:05:48 -08:00
Tim Abbott 5a02b33f2e digest: Add a large block comment on correctness. 2021-01-17 11:37:59 -08:00
Steve Howell 1040fb7219 email digests: Remove handle_digest_email shim.
The previous commit made it so we only call the
shim in tests, so now we completely remove it.
2021-01-17 11:28:30 -08:00
Steve Howell bfa0bdf3d6 email digests: Process users in chunks of 30.
This should make the queue empty more quickly,
because we do bulk queries to prevent database
hops.
2021-01-17 11:28:30 -08:00
Steve Howell e0b451730a email digests: Extract get_new_streams.
This makes us more efficient when handling
multiple users.  We don't have to keep
sending the same two queries to the database.

Note that as part of this we eliminated
a failure mode for the obscure population
of users from whom both `user.is_guest` and
`user.can_access_public_streams()` returns
False.  We know this would have only affected
Zephyr users (by looking at the code), and
we know we don't actually process Zephyr
users for email digests (or else we would
have raised exceptions in the old code).
2021-01-17 11:28:30 -08:00
Steve Howell 23de94504f email digests: Query streams for messages up front.
This should save us many hops to the database when
we process users in bulk.
2021-01-17 11:28:30 -08:00
Steve Howell 3662bf2dcb minor: Rename stream_map -> user_stream_map. 2021-01-17 11:28:30 -08:00
Steve Howell 11c93aced5 minor: Rename user_profile -> user and avoid shadowing. 2021-01-17 11:28:30 -08:00
Steve Howell f8bbb7fea9 email digests: Use select_related("realm").
We mostly need realm_id, but when we go to build
message lists, we need realm.uri.

We could probably be more aggresive about using
`only` here, but for now I am just trying to
reduce hops to the database.
2021-01-17 11:28:29 -08:00
Steve Howell bb56f0ec0e minor: Move get_stream_map to module level.
This is a pure code move.
2021-01-17 11:28:29 -08:00
Steve Howell 52e2d5a733 email digests: Avoid long_term_idle check.
We want to exclude users with recent subscription
activity from emails, regardless of whether
the long_term_idle flag is set.
2021-01-17 11:28:29 -08:00
Steve Howell 162b372b93 email digests: Do one query for recent streams.
This is another way to limit hops to the database
when we process users in bulk.
2021-01-17 11:28:29 -08:00
rht 16583fd5b4 docs: Say that certbot wildcard has been released. 2021-01-17 11:17:09 -08:00
Alex Vandiver c2526844e9 worker: Remove SignupWorker and friends.
ZULIP_FRIENDS_LIST_ID and MAILCHIMP_API_KEY are not currently used in
production.

This removes the unused 'signups' queue and worker.
2021-01-17 11:16:35 -08:00
Alex Vandiver 01658e39a9 sentry: Verify version is supported, first.
Raven SDK does not send a `title` field.
2021-01-17 11:15:40 -08:00
Alex Vandiver d688e18de2 errors: Remove references to "deployment", use "host".
The `deployment` key was only set in `do_report_error`, which is now
only used in one codepath (the queue worker).  The logging handlers on
staging call notify_server_error directly, which omits the
`deployment` key.

Remove the odd one-of key, and instead simply do dispatch in
`do_report_error`.
2021-01-17 11:08:12 -08:00
Aman Agrawal 9b23e540df message_edit: Stop stream_header_colorblock from overlapping others. 2021-01-17 11:07:32 -08:00