Commit Graph

39353 Commits

Author SHA1 Message Date
Steve Howell 73e7f7edec check-node-fixtures: Compare python/openapi schemas.
We might want to rename the tool to something more
general now, since we are really reconciling three
things:

    - node fixtures
    - event_schema checkers for test_events
    - openapi specs

The way we compare python and openapi schemas is
as follows:

    - first convert openapi schemas to be build
      from DictType, ListType, etc. with from_opeapi

    - do a diff on the schemas

Most of the new code is just having the FooType
family of classes serialize themselves with schema().
2020-09-25 11:43:19 -07:00
Steve Howell 2b76eb767f event_schema: Change propagate_mode to an enum. 2020-09-25 11:43:19 -07:00
Steve Howell aca641a4d1 refactor: Extract data_types module.
Defining types with an object hierarchy
of type classes will allow us to build
functionality that was impossible (or
really janky) with the validators.py
approach of composing functions.

Most of the changes to event_schema.py
were automated search/replaces.

This patch doesn't really yet take
advantage of the new FooType classes,
but we will use it soon to audit our
openapi specs.
2020-09-25 11:43:19 -07:00
Alex Vandiver 36ea307fbf puppet: Depend other changes on sharding.py validation.
Use the validation of the tornado sharding config that
`stage_updated_sharding` does, by depending on it.  This ensures that
we don't write out a supervisor or nginx config based on a
bad (e.g. non-sequential) list of tornado ports.
2020-09-25 10:52:40 -07:00
Alex Vandiver c0e240277b tornado: Remove fingerprinting, write out .tmp files always.
Fingerprinting the config is somewhat brittle -- it requires either
custom bootstrapping for old (fingerprint-less) configs, and may have
false-positives.

Since generating the config is lightweight, do so into the .tmp files,
and compare the output to the originals to determine if there are
changes to apply.

In order to both surface errors, as well as notify the user in case a
restart is necessary, we must run it twice.  The `onlyif`
functionality cannot show configuration errors to the user, only
determine if the command runs or not.  We thus run the command once,
judging errors as "interesting" enough to run the actual command,
whose failure will be verbose in Puppet and halt any steps that depend
on it.

Removing the `onlyif` would result in `stage_updated_sharding` showing
up in the output of every Puppet run, which obscures the important
messages it displays when an update to sharding is necessary.
Removing the `command` (e.g. making it an `echo`) would result in
removing the ability to report configuration errors.  We thus have no
choice but to run it twice; this is thankfully low-overhead.
2020-09-25 10:52:40 -07:00
Anders Kaseorg 58808c2362 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:33:45 -07:00
Anders Kaseorg 3a39c6f67f dependencies: Downgrade simplebar to 5.2.1.
SimpleBar 6.0.0-beta.2 through -beta.6 are built with ES6 syntax (I
assume inadvertently: https://github.com/Grsmto/simplebar/issues/523),
and its latest tag has moved back to 5.2.1 anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:13:47 -07:00
Wes Galbraith 9645959ac4 populate_db: Add emoji reactions to development environment database.
This change adds automated generated emoji reactions to the data in
the development environment's database.

Fixes part of #14991.
2020-09-23 16:10:37 -07:00
Steve Howell f29b2884ca bitbucket2: Format user info consistently.
We now use get_user_info() to format all of our
users in messages.
2020-09-23 15:31:38 -07:00
Steve Howell e0b6619dac bitbucket2: Simplify how we display user for fork events.
Even before GDPR changes, it was strange that we displayed
users differently for fork events vs. all other events.

After GDPR, we don't even get the `username` field any
more.

So now we simply use `display_name` if available, and then
we try `nickname`.

See https://developer.atlassian.com/cloud/bitbucket/bitbucket-api-changes-gdpr/
for more context.
2020-09-23 15:31:38 -07:00
Steve Howell 1ef8d79352 bitbucket: Decouple BITBUCKET_FORK_BODY between versions.
We were trying to share the same format string between
the two different versions of bitbucket, but this only
creates confusion, as the two versions are only close
enough to be confusing.

The format string might be the same, but the semantics
are different, as well as the eventual outputs.

For example, the {username} piece here is simple in version
2, but in version 3 we append a url to the user's name.
2020-09-23 15:31:38 -07:00
Alex Vandiver b06253049a stripe: Change `assert_called` to `assert_called_once`.
Prefer using `assert_called_once` to protect against places where a
mock might be reused, and in so doing have been previously called,
thus making the second usage of `assert_called` not assert anything of
note.
2020-09-23 15:29:47 -07:00
Alex Vandiver ab4f6b54ea stripe: Do not log credit card issues as errors.
Problems with the card itself should not be logged as errors -- while
perhaps notable in aggregate, they are not worthy of being logged to
Sentry, for instance.

Downgrade these to `info`; continue to log other problems at the
`error` level.  This updates tests for this change, and in so doing
corrects a test that does not do its job, due to a missing
`reset_mock`.
2020-09-23 15:29:47 -07:00
sahil839 fe370debe5 tests: Rename stream messages tests in test_message_send.py.
This commit renames 'test_message_to_self' and
'test_api_message_to_self' tests to
'test_message_to_stream_by_name' and
'test_api_message_to_stream_by_name' to depict
the actual purpose of these tests.
2020-09-23 15:28:31 -07:00
Aman Agrawal 5ae2325979 events: Don't send presence data for web public guests.
We disable presence and hide list of users in right sidebar
via setting realm_presence_disabled to false here for
web public guests.
2020-09-23 12:11:22 -07:00
Aman Agrawal 5153a036f2 events: Disable topic edit & msg edit/delete for web public guest.
To ensure web public guests cannot change any data, it's natural
that we disable topic & msg edit / delete ops.
2020-09-23 12:11:22 -07:00
Aman Agrawal 3ec23e1a9d fetch_initial_state_data: Handle case of web public guests.
user_profile will be None for web_public_guests here.  Hence, for
settings (of which most be inaccessible by web public guest),
which require a user_profile, we either set an empty value for
them or set them to a default value. This will help render
the frontend or extend support to our clients without breaking
a lot of code.

Tweaked by tabbott to add many comments.
2020-09-23 12:11:15 -07:00
Aman Agrawal 9cabd8f9cb process_client: Don't update activity of unauthenticated users.
This allows wrapper `add_logging_data` to be used to add
client information for unauthenticated users (or web public guests).
2020-09-23 12:07:06 -07:00
Aman Agrawal 6012d3cff2 get_raw_user_data: Support acting_user=None.
This already had support for web_public_guest, fixed mypy
annotation.
2020-09-23 12:07:06 -07:00
Aman Agrawal c88b4cba60 format_user_row: Support acting_user=None. 2020-09-23 12:07:06 -07:00
Aman Agrawal 48492a0633 fetch_initial_state_data: Pass realm as independent parameter.
This removes dependency of the function on user_profile to get the
realm, which will be useful when user_profile is None in case of web
public guests.
2020-09-23 12:06:54 -07:00
Gittenburg 6e95809cc4 emoji_picker: Fix search input length.
This was probably a regression from our upgrade to bootstrap 2.3.2.
2020-09-23 11:33:27 -07:00
Anders Kaseorg 9238813135 js: Use destructuring for require statements.
This allows import/order to auto-fix blocks including these
statements.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-23 09:06:07 -04:00
Anders Kaseorg 5216b70691 lint: Remove custom rule already enforced by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 18:11:21 -07:00
Anders Kaseorg 424689acdd tslint: Remove tslint.json.
We use ESLint, not TSLint; TSLint is deprecated in favor of ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 17:09:37 -07:00
Graham Bleaney 273f6f69e5 pysa: Update .pyre_configuration to point to typeshed.
This change adds a 'typeshed' entry to the '.pyre_configuration' file
to ensure that Pysa has access to type stubs during analysis.
2020-09-22 15:44:47 -07:00
Graham Bleaney 8449ac2836 dependencies: Add dependency on pyre-check for running Pysa.
This commit adds a dev dependency on the pyre-check package, to
enable the running of Pysa (a python static analyzer for security) in
integration tests.
2020-09-22 15:44:47 -07:00
sahil839 2381e25438 invites: Display name of referrer instead of email in invites list.
We now display the name of referrer instead of email in invites list
and clicking on the name opens the user popover.
This helps us to avoid showing fake emails when the email address
visibility is hidden.

Tweaked by tabbott to still look at both email and name for filtering.
2020-09-22 15:42:53 -07:00
sahil839 231af0f84d popovers: Make a single click handler for opening bot owner popover.
We remove handle_bot_owner_profile function and we handle the opening
of popover of bot owner from a single click handler in popovers.js
using 'view_user_profile' class.

We also rename 'view_user_profile' class to 'view_full_user_profile'
for the button in popover, which is used to open full user profile.
2020-09-22 15:42:53 -07:00
sahil839 4dc3e5f96f popovers: Enable keyboard support for user info popovers.
This commit enables keyboard support for user info popovers for
navigating through popover options using up/down keys.

We add get_user_info_popover_items function, whose implementation
is different from other similar functions. Instead of using
popover_data.$tip we directly use $("div.user-info-popover")
because when we open the popover of bot owner from the bot
popover, the element which opens the popover is removed from
DOM and popover_data is undefined.
2020-09-22 15:42:53 -07:00
sahil839 7525642507 popover: Show normal popover instead of extended profile one for bot owner.
We should show normal popover instead of extended profile one for the bot
owner in bots section of organization settings.

A new function show_user_info_popover is added, as it makes sense to keep
it separated from the function used to open popover for sender of a
message, which uses the message from which the popover is opened.
This added function can further be used for showing popover for
"invited_by" in invites table.
2020-09-22 15:42:53 -07:00
sahil839 5ed9e8f697 popover: Add "Copy mention syntax" option for non-message user popovers.
This commit replaces the "Reply mentioning user" option with "Copy mention
syntax" for user info popovers that are not opened from a message.

Clicking on "Copy mention syntax" will copy the mention syntax of user to
clipboard.

This change is done because user popovers not opened from message are not
linked to any message.
2020-09-22 15:42:53 -07:00
sahil839 fed6b283e6 hotkey: Check for open popovers before overlays for escape key.
We check for open popovers before overlays on pressing escape key
because we will be adding popovers in overlays for bot owners in
further commits also and we would want to close the popover only
on pressing escape key and not the overlay.
2020-09-22 15:42:53 -07:00
sahil839 569ef48699 popovers: Rename data-owner-id and data-bot-owner-id to data-user-id.
We rename data-bot-owner-id and data-owner-id, used to open user
profile of bot owners, to data-user-id such that we can make a
global click handler for all of them by making a separate class
in next commit.
2020-09-22 15:42:53 -07:00
sahil839 fc8ceceb1d popover: Rename keyboard handling functions for message user popovers.
We rename user_info_popover_handle_keyboard and get_user_info_popover_items
to user_info_popover_for_message_handle_keyboard and
get_user_info_popover_for_message items to differentiate it from functions
that will be added for bot-owner popovers.
2020-09-22 15:42:53 -07:00
Anders Kaseorg 66ce8903d3 urls: Remove raw string syntax on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 15:28:12 -07:00
Alex Vandiver fd20e54c79 webhooks: Never log JsonableError to webook loggers.
These represent known errors in what the user submitted.  This is
slightly complicated by UnsupportedWebhookEventType being an instance
of JsonableError.
2020-09-22 15:11:48 -07:00
Alex Vandiver 7001004ec0 webhooks: Do not predicate on the "payload" key.
If we are to log to the webhook logger, do so no matter which
arguments are passed.
2020-09-22 15:11:48 -07:00
Alex Vandiver 1a763696f7 webhooks: Only enable webhook logging if it is a webhook.
allow_webhook_access may be true if the request allows webhook
requests, regardless of if it only used for a webhook integration.

Only actually log to the verbose webhook logger if it is explicitly a
webhook endpoint, as judged by `webhook_client_name`.  This prevents
requests for `POST /api/v1/messages` from being logged to the webhook
logger if they mistakenly contain a `payload` argument.
2020-09-22 15:11:48 -07:00
Alex Vandiver 77d1a4a5c0 webhooks: Simplify logic around is_webhook_access.
We clearly allow webhook access if we are setting the
webhook_client_name.  This removes the need for the `or`s later.
2020-09-22 15:11:48 -07:00
Alex Vandiver d24869e484 webhooks: Rename is_webhook to allow_webhook_access.
This argument does not define if an endpoint "is a webhook"; it is set
for "/api/v1/messages", which is not really a webhook, but allows
access from webhooks.
2020-09-22 15:11:48 -07:00
Alex Vandiver b515c2bcbf webhooks: Add a missing format for the realm. 2020-09-22 15:11:48 -07:00
Aman Agrawal 1b5b82e712 RealmFilterPattern: Mark converted content as AtomicString.
If multiple filters match the same string, we run into an infinite
loop of converting string into urls. To fix it, we mark the matched
string as atomic after first conversion.
2020-09-22 15:10:38 -07:00
Anders Kaseorg 0228acf0f5 rest: Add rest_path shortcut for path with rest_dispatch.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg 5297e4a30a urls: Use unqualified imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg e70f2ae58d rest: Specify rest_dispatch handlers by function, not by string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Anders Kaseorg faf600e9f5 urls: Remove unused URL names and shorten others.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Mateusz Mandera f078509105 saml: Extract logic for determining if subdomain is allowed by an attr.
This logic has some trickiness due to the need to handle root subdomain
aliases, so this is worth extracting into a helper for use in other
backends.
2020-09-21 22:58:59 -07:00
Mateusz Mandera a0dea84edb saml: Add defensive code against org_membership attr not being a list. 2020-09-21 22:58:59 -07:00
Alex Vandiver db8daf4175 linkifiers: Allow tildes in target URLs. 2020-09-21 21:04:02 -07:00