Commit Graph

41889 Commits

Author SHA1 Message Date
Gaurav Pandey 303e7b9701 ci: Add Debian bullseye to production test suite. 2021-04-15 21:38:31 -07:00
Gaurav Pandey feb720b463 install: Add beta support for debian bullseye for production.
This won't work on a real bullseye system until Bullseye actually
officially releases.

Fixes part of #17863.
2021-04-15 21:38:31 -07:00
Gaurav Pandey 78524d4f87 provision: Add support for debian bullseye.
Fixes part of #17863.
2021-04-15 21:38:31 -07:00
Alex Vandiver 9de35d98d3 puppet: Ensure a snakeoil certificate, for Postfix and PostgreSQL.
We use the snakeoil TLS certificate for PostgreSQL and Postfix; some
VMs install the `ssl-cert` package but (reasonably) don't build the
snakeoil certs into the image.

Build them as needed.

Fixes #14955.
2021-04-15 21:37:55 -07:00
Anders Kaseorg bdb20a8002 integrations: Convert deprecated Django url to path.
django.conf.urls.url is actually a deprecated alias of
django.urls.re_path, but we want path instead of re_path.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg 2939d29b6d python: Convert deprecated Django smart_text alias to smart_str.
django.utils.encoding.smart_text is a deprecated alias of
django.utils.encoding.smart_str as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg dcdb00a5e6 python: Convert deprecated Django is_safe_url.
django.utils.http.is_safe_url is a deprecated alias of
django.utils.http.url_has_allowed_host_and_scheme as of Django 3.0,
and will be removed in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg e7ed907cf6 python: Convert deprecated Django ugettext alias to gettext.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Steve Howell 173ce9a3fc refactor: Use sub_store for get/validation.
This reduces the complexity of our dependency graph.

It also makes sub_store.get parallel to message_store.get.
For both you pass in the relevant id to get the
full validated object.
2021-04-15 17:26:17 -07:00
Steve Howell d2bbb7d521 refactor: Extract sub_store module.
This is a prep commit to break dependencies.

The sub_store module is a thin layer on top of
our map of stream_id -> sub.
2021-04-15 17:26:17 -07:00
Steve Howell 12650c1bec refactor: Extract stream_topic_history_util.
This breaks an indirect dependency of stream_data
on the channel module.

It's a verbatim code move, apart from the one-line
helper `has_history_for`. It's not totally clear
to me why the original code doesn't call into
`is_complete_for_stream_id` to early-exit, but
figuring that out is outside the scope of my
change.

It's possible that we will eventually just subsume
this tiny module into topic_list once we finish
breaking all dependencies, but we may want to
reuse this for something like Recent Topics
or other similar UIs.

It's also possible that we'll want to rename
stream_topic_history -> stream_topic_history_data
sometime soon, possibly after we clean up its
dependency on message_util soon.
2021-04-15 17:26:17 -07:00
Adam Birds 545cd961f4 integrations: Add docs for GitHub Actions integration.
I have added a documentation page for the GitHub Actions integration to
`/integrations/doc/github-actions` with a link to the Zulip GitHub
Actions repository.

Tweaked by tabbott to add cross-links with the main GitHub integration.
2021-04-15 16:42:31 -07:00
Adam Birds 576702fbeb
integrations: Update generated bot avatars.
I have updated any missing bot avatars after running the tool whilst
creating integrations. These needed adding otherwise the tool creates
them whenever it is ran.
2021-04-15 16:37:31 -07:00
Tim Abbott ae86ae9c4a i18n: Update translation data from Transifex. 2021-04-15 15:37:33 -07:00
Tim Abbott b40e50f295 settings: Fix elements incorrectly labeled as searches. 2021-04-15 15:36:05 -07:00
Tim Abbott 208721b3d7 left sidebar: Fix misleading "search streams" label.
This widget only filters the user's subscription -- it's only suggest
public streams that the user is not subscribed to.  "Filter" is the
correct label for a widget with this use case.
2021-04-15 15:36:05 -07:00
Vishnu KS 8362865c8d actions: Rename kandra-ops to kandra ops in oneclick config file.
This bug was introduced in f00c13d303
when we started to hardcode stream name in config file instead of
storing it as one of the GitHub secrets.
2021-04-15 10:41:20 -07:00
Aman Agrawal e41fffc43e pm_list: Show correct unread unread counts.
We only update the `.private_messages_header` here since
unread_counts of `.expanded_private_message` are updated
via `pm_list.update_private_messages`.

This fixes the bug of PMs in `.expanded_private_message` having
the same unread count as `private_messages_header`.

Since we rerender the DOM of `.expanded_private_message` every
time we update unread count of PMs, we don't need to manually
update them here. Also, we always keep them on display since
there is no real need to toggle them. They are not visible
when they have 0 unread counts via `.zero_count`.
2021-04-15 10:08:50 -07:00
Aman Agrawal e5acbf9498 activity: Fix buddy_list unread count not being updated instantly.
While rest of the app has ported to the new system of updating
unread_counts `activity` was not ported. This resulted in
unread count in buddy list not being updated when new
PMs arrive.
2021-04-15 10:08:50 -07:00
Steve Howell 06af1715cb bug fix: Fix today's regression with topic counts.
The series of commits to consolidate CSS classes
for the various unread-count spans across our app
created a bug where the stream_list.js code's selector
starting capturing the unread spans in topic list items.

Suppose you had a stream with these topics:

    Foo 10
        a 3
        b 3
        c 4

If another unread came in, you would briefly see:

    Foo 11
        a 11
        b 11
        c 11

Now we just use subscription_block to find the
element that we want to tweak.

I remove a convoluted node test here. Part of the
reason the node test was convoluted was that the
original implementation was overly complex. I will
try to re-introduce a simpler test soon, but this
is a bit of an emergency fix.
2021-04-14 16:29:49 -07:00
Steve Howell 2126478867 refactor: Simplify recent_senders code.
This reduces our dependency on message_list code (via
message_util), and it makes moving streams/topics and
deleting messages more performant.

For every single message that was being updated or
deleted, the previous code was basically re-computing
lots of things, including having to iterate through
every message in memory to find the messages matching
your topic.

Now everything basically happens in O(1) time.

The only O(N) computation is that we now lazily
re-compute the max message id every time you need it
for typeahead logic, and then we cache it for
subsequent use. The N here is the number of messages
that the particular sender has sent to the particular
stream/topic combination, so it should always be quite
small, except for certain spammy bots.

Once the max has been calculated, the common operation
of adding a message doesn't invalidate our cached
value. We only invalidate the cache on deletes.

The main change that we make here from a data
standpoint is that we just keep track of all
message_ids for all senders. The storage overhead here
should be negligible.  By keeping track of our own
messages, we don't have to punt to other code for
update/delete situations.

There is similar code in recent_topics that I think can
be improved in similar ways, and it would allow us to
eliminate functions like this one:

    export function get_messages_in_topic(stream_id, topic) {
        return message_list.all
            .all_messages()
            .filter(
                (x) =>
                    x.type === "stream" &&
                    x.stream_id === stream_id &&
                    x.topic.toLowerCase() === topic.toLowerCase(),
            );
    }
2021-04-14 16:28:07 -07:00
Tim Abbott 007a5bb95e devtools: Delete unnecessary user creation links.
These have a better UI on the main login page, and also don't work due
to the wrong HTTP method.
2021-04-14 15:28:00 -07:00
Anders Kaseorg f4d902e0cd i18n: Ignore missing translation errors from FormatJS.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-14 14:17:44 -07:00
Mateusz Mandera b4542cc059 message_edit: Verify the message is in a stream in move message API.
This wasn't being validated before. There wasn't any possibility to
actually succeed in moving a private message, because the codepath would
fail at assert message.is_stream_message() in do_update_message - but we
should have proper error handling for that case instead of internal
server errors.
2021-04-14 12:37:34 -07:00
Mateusz Mandera 0c0e83eaff message_edit: Verify user has access to old stream when moving message.
Otherwise an admin can move a topic from a private stream they're no
longer a part of - including the newest messages in the topic, that
they're not supposed to have access to.
2021-04-14 12:37:34 -07:00
Tim Abbott 9d852870ee streams: Delete risky helper get_stream_by_id. 2021-04-14 12:37:34 -07:00
Tim Abbott 7856098bdb archive: Use access_web_public_stream helper.
This is just a refactor to delete potentially security-sensitive
duplicated code.
2021-04-14 12:37:34 -07:00
Mateusz Mandera 3ba8348c51 CVE-2021-30487: Prevent admins from moving topics to disallowed streams.
A bug in the implementation of the topic moving API resulted in
organization administrators being able to move messages to streams they
shouldn't be allowed to - private streams they weren't subscribed to and
streams in other organization hosted by the same Zulip installation.

In our current model realm admins can't send messages to private streams
they're not subscribed to - and being able move messages to a
stream effectively allows to send messages to that stream and thus the
two need to be consistent.
2021-04-14 12:37:34 -07:00
Mateusz Mandera 140655d69e CVE-2021-30479: Prevent guest user access to all_public_streams API.
A bug in the implementation of the all_public_streams API feature
resulted in guest users being able to receive message traffic to public
streams that should have been only accessible to members of the
organization.
2021-04-14 12:37:34 -07:00
Mateusz Mandera 6e11754642 CVE-2021-30478: Prevent API super users from forging messages to other organizations.
A bug in the implementation of the can_forge_sender permission
(previously is_api_super_user) resulted in users with this permission
being able to send messages appearing as if sent by a system bots,
including to other organizations hosted by the same Zulip installation.

- The send message API had a bug allowing an api super user to
  use forging to send messages to other realms' streams, as a
  cross-realm bot. We fix this most directly by eliminating the
  realm_str parameter - it is not necessary for any valid current use
  case. The email gateway doesn't use this API despite the comment in
  that block suggesting otherwise.
- The conditionals inside access_stream_for_send_message are changed up
  to improve security. They were generally not ordered very well,
  allowing the function to successfully return due to very weak
  acceptance conditions - skipping the higher importance checks that
  should lead to raising an error.
- The query count in test_subs is decreased because
  access_stream_for_send_message returns earlier when doing its check
  for a cross-realm bot sender - some subscription checking queries are
  skipped.
- A linkifier test in test_message_dict needs to be changed. It didn't
  make much sense in the first place, because it was creating a message
  by a normal user, to a stream outside of the user's realm. That
  shouldn't even be allowed.
2021-04-14 12:37:34 -07:00
Mateusz Mandera 4235be759d CVE-2021-30477: Prevent outgoing webhook bots from sending arbitrary messages to any stream.
A bug in the implementation of replies to messages sent by outgoing
webhooks to private streams meant that an outgoing webhook bot could be
used to send messages to private streams that the user was not intended
to be able to send messages to.

Completely skipping stream access check in check_message whenever the
sender is an outgoing webhook bot is insecure, as it might allow someone
with access to the bot's API key to send arbitrary messages to all
streams in the organization. The check is only meant to be bypassed in
send_response_message, where the stream message is only being sent
because someone mentioned the bot in that stream (and thus the bot
posting there is the desired outcome). We get much better control over
what's going by passing an explicit argument to check_message when
skipping the access check is desirable.
2021-04-14 12:37:34 -07:00
Alex Vandiver ce0a90da37 docs: Update changelog for 3.3/3.4 releases. 2021-04-14 12:23:55 -07:00
Tim Abbott 85a7325757 css: Delete orphaned starred messages CSS.
This CSS stopped being used in
84afc67369.
2021-04-14 11:50:07 -07:00
Tim Abbott b7cfefc213 css: Add custom styling for starred message counts.
We use an inverted color scheme to what we use for unread messages, so
that one's eyes scan these as different from unreads.

We also need to introduce a 1px offset because the border takes up space.

Fixes #17938.
2021-04-14 11:43:38 -07:00
Aman Agrawal 97496088c9 recent_topics: Use `unread_counts` class for showing unread count. 2021-04-14 10:57:16 -07:00
Aman Agrawal 6744522229 user_presence_row: Use `unread_counts` class for showing unread count. 2021-04-14 10:57:16 -07:00
Aman Agrawal ab87325fea pm_list_item: Use `unread_counts` class for showing unread count. 2021-04-14 10:57:16 -07:00
Aman Agrawal 019afcd40d topic_list_item: Use `unread_counts` class for showing unread count. 2021-04-14 10:57:16 -07:00
Aman Agrawal b6d9577b48 stream_list: Use `unread_counts` class for showing unread count. 2021-04-14 10:57:16 -07:00
Aman Agrawal 84afc67369 top_left_corner: Directly use `span.unread_count` to display unreads.
In an effort to use a common class to display unread counts across
the app, we simplify the elements used to show unreads and use a
single `span` with `unread_count` class to do so.
2021-04-14 10:57:16 -07:00
Aman Agrawal 79bf740dcb top_left_corner: Extract function which updates unread count.
This is a common element that is being used both by pm_list and
top_left_corner, hence extraction makes sense.
2021-04-14 10:57:16 -07:00
Aman Agrawal b1f8041c31 giphy: Use simplebar to replace browser scrollbar. 2021-04-14 10:50:47 -07:00
Aman Agrawal 35bd44ed2a giphy: Add clear search button.
Make the input more square looking to match with our other
input boxes.
2021-04-14 10:50:47 -07:00
Aman Agrawal 84f7bb85f6 giphy: Make footer background black to match to with attribution.
Adjust border-radius to make edges smooth.
2021-04-14 10:50:47 -07:00
Aman Agrawal caa9720064 giphy: Directly use `Number` value of the breakpoint. 2021-04-14 10:50:40 -07:00
Aman Agrawal 802c450b3f realm: Add setting to configure GIPHY rating.
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
2021-04-14 10:29:39 -07:00
Aman Agrawal f9e0ed98b8 buddy_list_tooltip: Directly trigger tooltip on parent of elements.
Instead of listing all the elements in `.selectable_sidebar_block`,
we directly use it to trigger showing tooltip.
2021-04-14 10:15:58 -07:00
Aman Agrawal 6271a7db9c message_controls: Don't set css properties for all `div`s in it.
This tends to behave badly when we add new elements which don't
want these properties.
2021-04-14 10:15:58 -07:00
Signior-X 0a4eabd3e0 message-edit: Show the edited content on editing message failure.
This commit replaces the "echo_data.orig_raw_content" with
"echo_data.raw_content" in "message_edit.js". If the user is editing
a message and it fails, all the edited data used to get lost. Now, the
new edited content appears so that the user does not lose his edited
content.

This bug has been present since the original #12145, where a
refactoring error resulted in using the pre-edit data.

Fixes #18142.
2021-04-14 09:42:22 -07:00
Steve Howell 50bbbaea6a emoji typehahead: Fix render_emoji.
If you have two emojis of the same name, we were
trying to render the non-realm emoji without an
emoji_code, since the condition that I changed
here was erroneously just seeing if the name was
among the realm emojis.

This fix prevents a funny symptom that is reported
in #18135.  The symptom is that the non-realm
emoji causes us to render the top-left emoji in
our sprite sheet, since we didn't put emoji_code
into the args.

This doesn't fix the fundamental problem that
having two emojis of the same name is extemely
confusing. Our markdown parser obviously can't
distinguish ":thank_you:" from ":thank_you:".  For
the the typeahead case I suppose we could add some
kind of suffix.

So, this fix only simplifies debugging; it doesn't
get to the root of the problem.
2021-04-14 09:35:31 -07:00