Commit Graph

33059 Commits

Author SHA1 Message Date
Wyatt Hoodes c48e53f587 test_mattermost_importer: Fix missing coverage. 2019-08-20 15:23:41 -07:00
Mateusz Mandera 26f7dd09ce display_recipient: Eliminate raw_display_recipient from message dicts.
This was used as a helper to construct the final display_recipient when
fetching messages. With the new mechanism of constructing
display_recipient by fetching appropriate users/streams from the
database and cache, this shouldn't be needed anymore.
2019-08-20 12:22:52 -07:00
Tim Abbott b25abf3de9 display_recipient: Deduplicate list of display_recipient fields. 2019-08-20 12:22:37 -07:00
Mateusz Mandera d200cb921d display_recipient: Only fetch the necessary fields from the database.
There is no need to fetch the entire Stream or UserProfile objects, as
only several fields are needed. We use Django's .values() method to only
get what's needed.

For UserProfiles, it means that we get from the queries are dictionaries
already in the display_recipient form (UserDisplayRecipient type) - so
we can remove the user_profile_to_display_recipient_dict function, as
there's no need for this UserProfile -> UserDisplayRecipient conversion
anymore.
2019-08-20 12:15:30 -07:00
Mateusz Mandera 87ccb4f6c7 types: Consistently use DisplayRecipientT for display_recipient objects.
Instead of having the rather unclear type Union[str,
List[UserDisplayRecipient]] where display_recipient of message dicts was
involved, we use DisplayRecipientT (renamed from DisplayRecipientCacheT
- since there wasn't much reason to have the word Cache in there), which
makes it clearer what is the actual nature of the objects and gets rid
of this pretty big type declaration.
2019-08-20 12:15:30 -07:00
Mateusz Mandera 3ba0a37a92 types: Define UserDisplayRecipient type using TypedDict.
Since the display_recipients dictionaries corresponding to users are
always dictionaries with keys email, full_name, short_name, id,
is_mirror_dummy - instead of using the overly general Dict[str, Any]
type, we can define a UserDisplayRecipient type,
using an appropriate TypedDict.

The type definitions are moved from display_recipient.py to types.py, so
that they can be imported in models.py.

Appropriate type adjustments are made in various places in the code
where we operate on display_recipients.
2019-08-20 12:15:30 -07:00
Mateusz Mandera c779bb1959 messages: Don't use display_recipient values from cached message_dicts.
The user information in display_recipient in cached message_dicts
becomes outdated if the information is changed in any way.

In particular, since we don't have a way to find all the message
objects that might contain PMs after an organization toggles the
setting to hide user email addresses from other users, we had a
situation where client might see inaccurate cached data from before
the transition for a period of up to hours.

We address this by using our generic_bulk_cached_fetch toolchain to
ensure we always are fetching display_recipient data from the database
(and/or a special recipient_id -> display_recipient cache, which we
can flush easily).

Fixes #12818.
2019-08-20 12:15:30 -07:00
Pragati Agrawal 48efd46bc6 realm_logo: Remove redundant `realm_logo_url` function.
This function is nothing but a redundant layer of call to
`get_realm_logo_url`.
2019-08-20 12:07:20 -07:00
Pragati Agrawal 7326acdc2b realm_logo: Refactor `get_realm_logo_url` function. 2019-08-20 12:07:20 -07:00
Pragati Agrawal 1da5001875 realm_logo: Refactor/Deduplicate the code to sent the events.
This also creates a function to get source and url for logo.
2019-08-20 12:07:20 -07:00
Vaibhav c20f1945d7 css: Nest .help-table inside .modal-body. 2019-08-20 12:00:22 -07:00
Vaibhav a3c6bd68f0 css: Use SCSS nesting for .hotkeys_table. 2019-08-20 12:00:22 -07:00
Vaibhav c9b8de9cc1 css: Use SCSS nesting for .informational-overlays. 2019-08-20 12:00:22 -07:00
Vaibhav ba0f377273 css: Reorder informational_overlays.scss so similar elems in same place.
Also some whitespace changes. (No separate commit required).
2019-08-20 12:00:22 -07:00
Tim Abbott 69505c30a4 mattermost: Handle users who aren't on any team correctly.
It's not clear to me how this is intended to work in Mattermost's
system in that they don't document this behavior, but some users have
`null` as their list of teams, and presumably are not meant to be
included in any team at all.
2019-08-19 16:06:39 -07:00
Tim Abbott e02806ccc5 request: Fix other syntax error on Xenial Python. 2019-08-19 15:58:12 -07:00
Tim Abbott 0435556fd5 request: Fix syntax error on Xenial Python. 2019-08-19 15:47:40 -07:00
Tim Abbott 3781a0dae2 urls: Reformat how we handle API-only mobile endpoints.
This restructures the API endpoints that we currently have implemented
more or less for exclusive use by the mobile and desktop apps (things
like checking what authentication methods are supported) to use a
system that can be effectively parsed by our test_openapi
documentation.

This brings us close to being able to eliminate
`buggy_documentation_endpoints` as a persistently nonempty list.
2019-08-19 15:34:40 -07:00
Tim Abbott 27bfed7896 openapi: Remove presence from untestable endpoints.
This add some regular expression manipulation hacks to make it
possible for us to validate the documentation for the presence
endpoint with a slightly more complex regular expression capture
group.
2019-08-19 15:17:28 -07:00
Tim Abbott 44db59bbe0 openapi: Fix handling of parameters passed via the URL/path.
Previously, our OpenAPI documentation validation was failing for some
endpoints because it didn't account for the `in: path` type of
parameter, resulting in a mismatch between what was declared via REQ
and what was declared in the OpenAPI docs.

We fix this by excluding the path type parameters in both places from
what's considered by documentation using the `path_only` flag.

I doubt this is the correct long-term fix; in particular, I don't
think we're actually running the validators for these path-only
parameters.  The examples that exist today are all IDs with validators
for being non-negative numbers, but longer-term I think we'll want to
do something different (possibly at the REQ layer, see the TODO).
2019-08-19 15:06:08 -07:00
Hemanth V. Alluri 2e20dcc053 webhooks/jira: Update webhook to support improved comment events.
Atlassian announced that it will no longer provide information about
comments along with their "issue" type event payloads for Jira. So
we must now update the Jira integration to appropriately respond to
"comment" type events (the reason why we didn't do this before was
that initially, the "comment" type event payloads didn't contain
sufficient information about their issues, but this payload has
since been improved).

Note: This commit does *not* remove support for the older "issue"
type event payloads where information about comments was included.
This way we can maintain compatibility with old self-hosted versions
of self hosted Jira (2016 and before).

Source:
https://developer.atlassian.com/cloud/jira/platform/change-notice-
removal-of-comments-from-issue-webhooks/

Fixes #13012
2019-08-19 14:22:42 -07:00
Tim Abbott ee63f68140 docs: Add links from SSL docs to our desktop app SSL guide.
This should help people setting up new Zulip servers discover how to
setup SSL properly for the desktop apps.
2019-08-19 14:21:56 -07:00
Vishnu Ks d472d30c28 requirements: Upgrade stripe pip package from 2.21.0 to 2.35.0. 2019-08-19 11:09:33 -07:00
Vishnu KS 6c8d837401 stripe: Don't check value of realm_id directly in tests.
Value of realm_id varies from one development environment
to another from time to time. So we will just check whether
realm_id is a number instead.
2019-08-19 11:09:33 -07:00
Alexandra Ciobica 96cdfd676b css: Add bottom margin to titles in the register pages. 2019-08-18 12:57:25 -07:00
Pragati Agrawal eedcdf7f3b settings/styles: Reorder `.settings-section .table-striped` to same place. 2019-08-18 12:44:40 -07:00
Pragati Agrawal 78d6ecfa08 settings/styles: Use `.settings-section` for settings `.table-striped`.
We have used `.settings-section .table-striped` for other rules to refer the same elements
which are referred by `#settings_page .table-striped`.
2019-08-18 12:44:40 -07:00
Pragati Agrawal aac9a7a4b1 settings/styles: Remove redundant rules for `.table-striped thead th`.
This rule is already specified at
 `.settings-section .table-striped thead th`.
2019-08-18 12:44:40 -07:00
Pragati Agrawal a858d51a85 settings/styles: Nest all `.settings-section` rules. 2019-08-18 12:44:40 -07:00
Pragati Agrawal 9acd8caa59 settings/styles: Remove redundant specificity. 2019-08-18 12:44:40 -07:00
Pragati Agrawal 2c846774c0 settings/styles: Reorder styles in more sensible order.
Rather just putting rules in any order it makes more sense to have an order
of basic to more advanced/specific CSS.
2019-08-18 12:44:40 -07:00
Pragati Agrawal 94d867cd11 settings/scss: Reorder `.settings-section` to have them in same place. 2019-08-18 12:44:40 -07:00
Pragati Agrawal b0b1435a18 settings/scss: Remove now redundant `settings-wrapper` class.
No class like this is rendered to page.
2019-08-18 12:44:40 -07:00
Pragati Agrawal 378c1a5994 settings/scss: Reorder to have `.settings-section-title` at same place. 2019-08-18 12:44:40 -07:00
Wyatt Hoodes 92dd5e2c4f test_queue_worker: Remove timing from test_slow_queries_worker.
Testing a specific query time was causing spurious failures
in CI.
2019-08-18 12:32:45 -07:00
Wyatt Hoodes 5ee7553214 popovers: Fix broken user popover behavior.
If we call `popovers.hide_all` with a smaller browser
window, this breaks the functionality that the
conditional is attempting to handle.  We instead use
`hide_all_except_sidebars` to prevent the user list
from being closed.

If the display setting to show the user list in the
left sidebar is enabled, the behavior is even worse.
We add a conditional to maintain the streamlist
sidebar when clicking the chevron to show and hide
the popover here as well.
2019-08-18 12:12:52 -07:00
Anders Kaseorg 349f22524e provision: Don’t rely on uid == gid assumption when invoking chown.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 12:59:09 -07:00
Anders Kaseorg fa913b7f5d node_cache: Avoid shelling out for rm, ln, mkdir, cp, cd, touch.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 12:59:09 -07:00
Anders Kaseorg 88793e80ea provision: Skip some rm, mkdir, chown commands if possible.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 12:59:09 -07:00
okmanl 2a1305de9f lint: Add a rule to avoid msgid as a Python variable name.
This is for consistency with our usual patterns, see #12995.  We will
need a similar commit for JavaScript to complete #12995.
2019-08-17 12:47:13 -07:00
Wyatt Hoodes dc191b0be4 test_queue_worker: Test actual code path for slow_queries.
Instead of just mocking some fake events, we use the code
path that generates slow query events and publishes them
to SlowQueryWorker.

This test improvement would have got a recent potential regression
caught in code review.
2019-08-17 12:38:02 -07:00
Brutus5000 a386e3aebb github webhook: For release events show actions and tag name. 2019-08-17 12:35:16 -07:00
Hemanth V. Alluri dac068df31 production: Finish adding production support for Zulip on Debian Buster.
This commit finishes adding end-to-end support for the install script
on Debian Buster (making it production ready). Some support for this
was already added in prior commits such as
99414e2d96.

We plan to revert the postgres hunks of this once we've built
tsearch_extras for our packagecloud archive.

Fixes #9828.
2019-08-17 12:22:32 -07:00
Hemanth V. Alluri 083723b6a9 puppet: Add Zulip specific postgres configuration for 11.
Based on the work done in a03e4784c7.
2019-08-17 11:41:11 -07:00
Hemanth V. Alluri 792283c441 puppet: Commit an upstream version of postgres 11 config.
In preparation for adding production support for Debian Buster. Based
on the work done in commit 964a1ac8a7.
2019-08-17 11:41:11 -07:00
Hemanth V. Alluri 5dd45b4b2e puppet: Fix the release detection regex patterns in base.pp.
The issue here was that the '.' character was unescaped and the
regex was not anchored with a terminal '$'. This was detected by
Anders Kaseorg.

Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 11:41:11 -07:00
Hemanth V. Alluri 243d8ffc51 openapi: Add a test to ensure generated curl examples work.
This new test runs each generated curl example against the Zulip API,
checking whether it returns successfully without errors.

Significantly modified by tabbott for simplicity.
2019-08-17 11:35:08 -07:00
Tim Abbott 26ac3ebd3e openapi: Suggest the -sS options to curl.
These options prevent curl from doing the downloading progress bar
(which is clutter).
2019-08-17 11:35:08 -07:00
Tim Abbott 2366490ffc openapi: Pass api_url to curl example generation.
Our new curl example generation logic was broken, in that it hardcoded
localhost:9991 (without an HTTP method or anything) as the API URL.

It requires a bit of plumbing to make this possible.
2019-08-17 11:35:08 -07:00
Tim Abbott e8b4e80d45 test-api: Fix double use of django.setup.
django.setup is already called (with different/better environment
variables) inside test_server_running; we shouldn't be calling it just
before that to make imports work.

I discovered this because imports done at the wrong time would
potentially incorrectly have `testserver` as the EXTERNAL_HOST.
2019-08-17 11:35:08 -07:00