Commit Graph

36260 Commits

Author SHA1 Message Date
Aman Agrawal 7197a7ac68 message_edit: Add support for changing stream of a message.
* This feature is currently only visible to admins.
* Locally echoed messages are also updated.
* Add UI for editing stream if user is admin.
* Show propagate mode selector if either stream or topic changed.
2020-05-11 16:25:47 -07:00
Steve Howell c4d0e4c9f9 bot_data: Eliminate all uses of `bot.owner` (email).
We now use `bot.owner_id` for everything internally.
2020-05-11 16:16:58 -07:00
Steve Howell 9f137c3a05 bots: Extract user_dropdown widget.
We use this new widget in bot settings panels
(personal and org).  It lets you re-assign a
bot to a new human user.

Ideally we can improve this code to use
our existing list widgets to make it more
performant for realms with lots of users.
2020-05-11 16:16:58 -07:00
Steve Howell 5c16bb9c99 bot settings: Load bots independently.
We no longer use `/json/users` in the codepath
for bot settings (admin side).

We also specifically don't load human users when
we load bots, so you no longer have to pay for
the server round trip as a side effect of loading
bots.  Instead, there is a dedicated `set_up_bots`
entry point.

We also get the bot ids directly from `bot_data` now.

This commit, to some degree, builds on the prior commit
that had us hydrate data from `people.js` instead
of the payload from `/json/users`.
2020-05-11 16:16:58 -07:00
Steve Howell f6b1176045 bot settings: Use `get_item` in the list widget.
Our `list_render` list widget gives us the
option to use ids as our "list" and then
hydrate that list on-demand with an
`opts.get_item` function.

We now use that for the bots list, passing
in `bot_info` as that option.

And, importantly, we are now actually
hydrating the bot data from `bot_data.js`
data structures, and not `/json/bots`.

Using the `get_item` scheme has a couple
benefits:

    - Our sort functions are based on the
      actual items that we use to build the
      template, so there's a bit less
      code duplication.  (Generally, the
      data that we pass in to the template
      is "finalized" in some sense, such
      as the bot owner name.)

    - We are less likely to display stale
      data.

    - We are less likely to wire up filters
      to intermediate data elements that are
      not actually displayed to users (think
      of email vs. delivery_email).

We do rely on `get_item` (i.e. `bot_info`)
to be inexpensive, which it should be.

Note that we haven't completely decoupled
ourselves from `/json/bots`, which we still
use as our source for bot user_ids.  We will
fix that in the next commit.
2020-05-11 16:16:58 -07:00
Steve Howell be064e6104 list_render: Add get_item option.
We want to move toward having list consumers
pass us in a list of ids that we hydrate later
in the process.  This should help live-update
scenarios.  The next commit will describe the
benefits in a bit more detail, using the
concrete example of our bot settings table
in the org settings.

A slightly longer-term goal here is to be
able to ask `list_render` to re-render a particular
id, and this moves us closer to that.  But even
before that, this change should eliminate a class
of bugs dealing with stale data, such as when
you manually patch a list (with direct jQuery
hacks) but then later go to sort/filter the rows.
We will now re-hydrate the items in those scenarios.
2020-05-11 16:16:58 -07:00
Steve Howell cea6214ce8 user settings: Remove reset/meta.loaded logic.
We don't really need to know whether we've loaded
the user-related panels, since we only used `meta.loaded`
for a tiny optimization to avoid a jQuery lookup.

We rely mostly on the list widgets from `list_render`,
and they are smart enough to repopulate themselves
when they're called subsequent times.
2020-05-11 16:16:58 -07:00
Steve Howell 155f6da8ba bots: Add owner_id to bot-related payloads.
For the below payloads we want `owner_id` instead
of `owner`, which we should deprecate.  (The
`owner` field is actually an email, which is
not a stable key.)

    page_params.realm_bots

    realm_bot/add

    realm_bot/update

IMPORTANT NOTE: Some of the data served in
these payloads is cached with the key
`bot_dicts_in_realm_cache_key`.

For page_params, we get the new field
via `get_owned_bot_dicts`.

For realm_bot/add, we modified
`created_bot_event`.

For realm_bot/update, we modified
`do_change_bot_owner`.

On the JS side, we no longer
look up the bot's owner directly in
`server_events_dispatch` when we get
a realm_bot/update event. Instead, we
delegate that job to `bot_data.js`.
I modified the tests accordingly.
2020-05-11 16:16:58 -07:00
Steve Howell 393551bf81 bot settings: Live-update w/owner name (not email).
This fixes the fact that we update the bot table
with the owner's email instead of a name, but as
the TODO indicates, this is not a full fix, since
I don't linkify the owner name.

To do the full fix properly, I want to make it
so that the `list_render` widgets can just be given
an id of a row to update, and that's coming soon,
hopefully.  If I get sidetracked, the ugly ways to
do this are one of the following:

    - just duplicate what the template does in
      jQuery

    - extract a partial to draw the bot owner link

The full solution here should fix ALL the live
update code in `update_user_data`, which is why
I'm hesitant to add any interim complexity.
2020-05-11 16:14:04 -07:00
Steve Howell 47f07eeb2e minor: Move update_user_data() down in file.
This is just a lexical change.  We are going
to use some shared code soon that we don't want
to export, and if `update_user_data()` is
declared too early in the file, then the function
we extract will either need to be exported (to
satisy the linter) or placed far away from its
most natural siblings.
2020-05-11 16:14:04 -07:00
Steve Howell 91fa64e8e6 refactor: Extract `bot_owner_full_name`.
We will use this for a patch to the live-update
code, and it also de-clutters `bot_info`.

This function could plausibly live in `people.js`,
but it's not worth the indirection at this time,
and, also, one of the upcoming callers to the
function will only temporarily need it.

There's a little bit of a chicken/egg problem
going on:

    - It's hard to have nice system-wide
      APIs related to bots while bot settings
      are still in flux.

    - It's hard to clean up the bot settings
      code while the system-wide API is still
      kinda messy.

But I'm making slow progress on that front.
2020-05-11 16:14:04 -07:00
Steve Howell c8fd4e01e1 minor: Remove obsolete `is_active_human`. 2020-05-11 16:14:04 -07:00
Steve Howell b746547395 user settings: Fix email sort for bots/deactivateds.
We were still sorting them with the generic alphabetic
sort due to the markup, despite passing in a custom
sort to the `list_render` class.

The `sort_email` helper often behaves like a generic
alphabetic sort, so this fix is mostly just making the code
do what it claims to do (and it's consistent with how
we already sort active users).

The nuance with emails is whether we display real
emails or system-generated emails.
2020-05-11 16:14:04 -07:00
Tim Abbott d3c7579730 gitlab: Fix missing coverage/bug for job hooks corner case.
Apparently, the change and test I added before didn't quite cover the
corner case that was broken.  This does, and exposes a second bug as
well, which we fix.
2020-05-11 16:13:23 -07:00
Tim Abbott 225bbf3633 puppet: Update check_postgres_replication_lag for postgres 10.
These functions were renamed in postgres 10.
2020-05-11 15:59:23 -07:00
Tim Abbott d4f875f7c5 webhooks: Add missing gitlab test fixture. 2020-05-11 15:04:34 -07:00
Rohitt Vashishtha 26238426cd message-edit: Process user groups for mentioned user ids.
When editing a message where we mention a usergroup, we would remove
the 'mentioned' flag from messages, resulting in the message being
hidden from your mentions in the UI. This was reported by Greg Price in
https://chat.zulip.org/#narrow/stream/9-issues/topic/missing.20mention.

We add the same code that we use in do_send_messages to calculate the
updated mentions_user_ids. We add some tests alongside other user group
mention tests in test_bugdown.
2020-05-11 14:52:11 -07:00
Puneeth Chaganti 614d4a3da8 docs: Add note about generating integrations bot avatar image. 2020-05-11 14:49:11 -07:00
Puneeth Chaganti 322e84ffeb integrations: Add ERROR prefix to missing bot avatar message. 2020-05-11 14:49:11 -07:00
Rohitt Vashishtha cf2fafa537 styles: Use pipe character as separator in navbar.
The previous implementation had a weird bug where for some streams, the 1px wide
before and ::after elements would appear to have different widths. See conversation:

https://chat.zulip.org/#narrow/stream/101-design/topic/navbar.20redesign/near/873312
2020-05-11 14:38:58 -07:00
Rohitt Vashishtha 032361c66d styles: Refactor to remove a duplicate block. 2020-05-11 14:38:58 -07:00
Tim Abbott 11027161c3 webhooks: Fix handling of GitLab Job Hook events.
This fixes an exception for these events.

Fixture from https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#job-events.
2020-05-11 14:37:01 -07:00
Rohitt Vashishtha 6baf95d88a settings: Expose update function from settings_list_widget. 2020-05-11 14:00:05 -07:00
Rohitt Vashishtha b066ba1ff3 settings: Remove settings_org dependency from settings_list_widget.
Instead of taking a subsection option and calling the settings_org
function to update that subsection, we now take a callback function
as on_update. Also, we now store the value initial value of the
widget in opts.value instead of reading again from page_params.

These changes allow us to use this widget outside of settings_org
and for values other than settings that are in page_params.
2020-05-11 14:00:05 -07:00
arpit551 f3ab8e66dc ci: add configuration for posting to czo whenever CI fails in master.
Since notify is mostly depreciated  in the latest versions of CircleCI.
Although we can use use notify in CircleCI 2.0 but currently
there is no documentation regarding it.

We could have use notify here rather than this hacky solution but
if we use notify it was not possible to trigger CircleCI webhook only
for the main Zulip repository.

Also corrected the circle ci webhook for the case where we don't receive
previous in post request
2020-05-11 13:45:22 -07:00
Rohitt Vashishtha e065f51824 docs: Remove duplicate content from deploying-bots.md. 2020-05-11 13:42:22 -07:00
Rohitt Vashishtha 2ad55ef11b slack-import: Update docs to reflect the removal of Slack legacy tokens.
This commit deatails how users can generate the new type of APi tokens
by creating a new slack app with the correct scopes specified.

Fixes #14963.
2020-05-11 13:41:50 -07:00
Rohitt Vashishtha 9506be0f4f slack-import: Downgrade Slack legacy-token check failure to warning.
Slack has disabled creation of legacy tokens, which means we have to use other
tokens for importing the data. Thus, we shouldn't throw an error if the token
doesn't match the legacy token format.

Since we do not have any other validation for those tokens yet, we log a warning
but still try to continue with the import assuming that the token has the right
scopes.

See https://api.slack.com/changelog/2020-02-legacy-test-token-creation-to-retire.
2020-05-11 13:41:50 -07:00
Tim Abbott 937930cc90 integrations: Add bot avatar for grafana.
This was missed because the requirement to add these was new since the
original PR.
2020-05-11 01:07:44 -07:00
Tim Abbott d8ea649869 puppet: Cast tornado_processes to Integer.
This is the latest mechanism in puppet for turning a string into an
integer.

We update an adjacent comment while we're at it.
2020-05-11 00:54:48 -07:00
Tim Abbott 6319c181eb puppet: Use actual name for the bind9-host package.
Using the `host` virtual package confused Puppet into reporting it was
doing work every time one did a puppet run, resulting in unnecessarily
spammy output.
2020-05-11 00:51:53 -07:00
Mateusz Mandera dd40649e04 queue_processors: Remove the slow_queries queue.
While this functionality to post slow queries to a Zulip stream was
very useful in the early days of Zulip, when there were only a few
hundred accounts, it's long since been useless since (1) the total
request volume on larger Zulip servers run by Zulip developers, and
(2) other server operators don't want real-time notifications of slow
backend queries.  The right structure for this is just a log file.

We get rid of the queue and replace it with a "zulip.slow_queries"
logger, which will still log to /var/log/zulip/slow_queries.log for
ease of access to this information and propagate to the other logging
handlers.  Reducing the amount of queues is good for lowering zulip's
memory footprint and restart performance, since we run at least one
dedicated queue worker process for each one in most configurations.
2020-05-11 00:45:13 -07:00
Jenny Ghose 180c16c80e integrations: Add an incoming webhook for Grafana.
Tweaked by tabbott to use formatted suggested in one of the various
duplicate PRs for this issue, showing the rule name clearly.

Fixes #12951.
2020-05-11 00:27:38 -07:00
Tim Abbott a0c2121958 docs: Advertise Slack-compatible webhook a bit.
This should make it discoverable enough that users will try it out and
send us feedback.
2020-05-11 00:07:28 -07:00
Chris Heald c80e913c7a webhooks: Add a webhook capable of parsing Slack payloads.
This adds a webhook that can be used to interpret standard Slack
payloads. Since there are a ton of existing Slack integrations out
there, having a webhook which can accept standard Slack payloads can
significantly ease transition pains. Obviously this can't do everything
that Slack payloads can (particularly WRT their widgets/interactions),
but we can ingest text and parse out multi-block payloads into a message
relatively reasonably.
2020-05-11 00:07:28 -07:00
Tim Abbott 21a04e2dbc puppet: Use nice to deprioritize various processes.
Our priority hierarchy is:
(1) Tornado and base services like memcached, redis, etc.
(2) Django and message sender queue workers.
(3) Everything else.

Ideally, we'd have something a bit more fine-grained (e.g. some queue
workers are potentially in the sending path, while others aren't), but
this should have a big impact on ensuring Tornado gets the resources
it needs during load spikes.

I think this has a good chance of causing some load spikes that would
previously have resulted in a user-facing delivery delays no longer
having any significant user-facing impact.
2020-05-10 23:28:25 -07:00
Siddharth Varshney d1d012c1c4 desktop-login: Use zulip design styles for desktop-login.
Fixes: #14827
2020-05-10 23:05:19 -07:00
shubhamgupta2956 9cd8644c7c uploads: Add support for ".jpe" file extension.
Currently when the user uploads files with ".jpe" file extension, the
markdown is converted to link but the image is not embedded.

This commit adds the support for ".jpe" file extension.

Fixes #14863
2020-05-10 22:55:52 -07:00
Steve Howell 63154b335e bot_data test: Remove is_admin noise.
We no longer use `is_admin` in `bot_data`.
2020-05-10 16:20:41 -04:00
Steve Howell f9e2e357bf bot_data test: Clean up test people.
The prior version of "me" confusingly had the same
user_id as one of our bots, so I fixed that.

I also avoid using a test email of 'owner@zulip.com',
which is confusing for earlier tests where I haven't
established "me" as the actual owner of any bots.
2020-05-10 16:20:41 -04:00
Steve Howell 7c1f64d4e5 bot_data: Remove get_bot_owner_email. 2020-05-10 16:20:41 -04:00
Pragati Agrawal 60a762704a org settings: Fix limited plan realm can change message_retention_days.
These changes should be included in bd9b74436c,
as it makes sure that Zulip limited plan realm won't be able to change the
`message_retention_days` setting.
2020-05-09 16:40:50 -07:00
Anders Kaseorg 78c70b1424 bugdown: Leave link titles alone until clean_user_content_links.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-09 16:32:40 -07:00
Anders Kaseorg 83a0006602 clean_user_content_links: Show the full URL in the title.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-09 16:32:40 -07:00
Anders Kaseorg 32f3fd1c77 bugdown: Fix ElementPair typing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-09 16:32:40 -07:00
Anders Kaseorg 6aaeab75bc bugdown: Fix ResultWithFamily typing.
It needs to be a full class because a generic NamedTuple doesn’t work
in Python 3.6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-09 16:32:40 -07:00
Steve Howell 2df183142c user settings: Flatten template data.
We now no longer have to remember that
`is_guest` is on `user` but `is_current_user`
is in `..`.

And we no longer have to remember that
`full_name` is on `user` but `display_email`
is in `..`.
2020-05-09 10:41:14 -07:00
Steve Howell 71c2bde665 user settings: Extract human_info(). 2020-05-09 10:41:14 -07:00
Steve Howell e9298dab5b bot settings: Extract bot_info().
We now gather all the bot info in one place, rather
than grabbing some of it during the triage phase and
then some of it later.

We also explicitly copy over the fields that we
need for the template, in preparation for two
efforts:

    - We want to get data from `people.js` and
      avoid the round trip to `<server>/json/users`.

    - We want to simplify the template by
      flattening our data.  (It's really somewhat
      arbitrary whether `is_admin` is a calculated
      value, for example, but we currently leak
      that implementation detail to the template.)

We can't flatten this data quite yet, since we
share the same template for bot users as human users,
so we'll fix the human data in a bit.
2020-05-09 10:41:14 -07:00
Steve Howell d45f0171cb user settings: Avoid status_field confusion.
We now close on status_field in our event handlers,
so that there's no chance of writing to the wrong
status field if somebody switches panels before
we have a status to report.

We can't eliminate `get_status_field` yet, but that
will go away in a future commit.
2020-05-09 10:41:14 -07:00