Commit Graph

35088 Commits

Author SHA1 Message Date
Anders Kaseorg a491c5a162 styles: Fix SimpleBar hover styling for simplebar@5.1.0.
Follow an upstream adjustment to the styling of the vertical
scrollbar (but not the horizontal scrollbar).

https://github.com/Grsmto/simplebar/issues/420

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-13 16:14:23 -07:00
Steve Howell fcc5ae5247 invites: Fix regression w/email vs. delivery_email.
In 220c2a5ff3 I
introduced a query to find invites by delivery_email
but was still using email as the key.

For most realms `email` and `delivery_email` are
synonymous, so this temporary bug would not affect
them.  For realms that restrict emails, the invite
would have probably failed for other reasons, but
the symptom would have been less clear.
2020-03-12 10:13:08 -04:00
majordwarf 68dcdcd28e todo_list: Add task description and index numbers.
The user can pass description along with the task name by splitting the input string with hyphen.

Eg: Task Title - Task Description
todo_list: Add index numbers to task.
2020-03-12 08:02:06 -04:00
majordwarf ba2f11f78c todo_list: Render task name in bold text. 2020-03-12 08:02:06 -04:00
Steve Howell 1b16693526 tests: Limit email-based logins.
We now have this API...

If you really just need to log in
and not do anything with the actual
user:

    self.login('hamlet')

If you're gonna use the user in the
rest of the test:

    hamlet = self.example_user('hamlet')
    self.login_user(hamlet)

If you are specifically testing
email/password logins (used only in 4 places):

    self.login_by_email(email, password)

And for failures uses this (used twice):

    self.assert_login_failure(email)
2020-03-11 17:10:22 -07:00
Steve Howell c235333041 test performance: Pass in users to api_* helpers.
This reduces query counts in some cases, since
we no longer need to look up the user again. In
particular, it reduces some noise when we
count queries for O(N)-related tests.

The query count is usually reduced by 2 per
API call.  We no longer need to look up Realm
and UserProfile.  In most cases we are saving
these lookups for the whole tests, since we
usually already have the `user` objects for
other reasons.  In a few places we are simply
moving where that query happens within the
test.

In some places I shorten names like `test_user`
or `user_profile` to just be `user`.
2020-03-11 14:18:29 -07:00
Steve Howell 626ad0078d tests: Add uuid_get and uuid_post.
We want a clean codepath for the vast majority
of cases of using api_get/api_post, which now
uses email and which we'll soon convert to
accepting `user` as a parameter.

These apis that take two different types of
values for the same parameter make sweeps
like this kinda painful, and they're pretty
easy to avoid by extracting helpers to do
the actual common tasks.  So, for example,
here I still keep a common method to
actually encode the credentials (since
the whole encode/decode business is an
annoying detail that you don't want to fix
in two places):

    def encode_credentials(self, identifier: str, api_key: str) -> str:
        """
        identifier: Can be an email or a remote server uuid.
        """
        credentials = "%s:%s" % (identifier, api_key)
        return 'Basic ' + base64.b64encode(credentials.encode('utf-8')).decode('utf-8')

But then the rest of the code has two separate
codepaths.

And for the uuid functions, we no longer have
crufty references to realm.  (In fairness, realm
will also go away when we introduce users.)

For the `is_remote_server` helper, I just inlined
it, since it's now only needed in one place, and the
name didn't make total sense anyway, plus it wasn't
a super robust check.  In context, it's easier
just to use a comment now to say what we're doing:

    # If `role` doesn't look like an email, it might be a uuid.
    if settings.ZILENCER_ENABLED and role is not None and '@' not in role:
        # do stuff
2020-03-11 14:18:29 -07:00
Steve Howell 00dc976379 tests: Use users for common_subscribe_to_streams.
We also use users for get_streams().
2020-03-11 14:18:29 -07:00
Sourabh Singh 1b3cfecf2a
webhooks: Add team reviewers support in github webhook.
The github webhook implementation previously ignored the "team reviewers"
part of pull_request events, resulting in inaccurate output.

Fixes: #14096.
2020-03-10 16:29:59 -07:00
Tim Abbott 68274cae74 apps: Fix broken desktop download links.
This was introduced by my failing to properly test the recent bundle
of changes to this logic.
2020-03-10 14:38:23 -07:00
BIraj 60385366f0 js: Cleanup debug.js example to use ES syntax. 2020-03-09 14:07:13 -07:00
Mateusz Mandera 2000608a9e report_error: Fix inaccurate docstring.
do_report_error isn't actually below.
2020-03-09 13:54:58 -07:00
Mateusz Mandera 89394fc1eb middleware: Use request.user for logging when possible.
Instead of trying to set the _requestor_for_logs attribute in all the
relevant places, we try to use request.user when possible (that will be
when it's a UserProfile or RemoteZulipServer as of now). In other
places, we set _requestor_for_logs to avoid manually editing the
request.user attribute, as it should mostly be left for Django to manage
it.
In places where we remove the "request._requestor_for_logs = ..." line,
it is clearly implied by the previous code (or the current surrounding
code) that request.user is of the correct type.
2020-03-09 13:54:58 -07:00
Mateusz Mandera 0255ca9b6a middleware: Log user.id/realm.string_id instead of _email. 2020-03-09 13:54:58 -07:00
akashaviator 700123a30b api: Document DELETE ../messages/{message_id}/reactions endpoint.
This refactors remove_reaction in python_examples.py to validate the
result with validate_against_openapi_schema.  Minor changes and some
additions have been made to the OpenAPI format data for
/messages/{message_id}/reactions endpoint.
2020-03-08 19:12:45 -07:00
akashaviator 5dd1a1fc83 api: Document POST ../messages/{message_id}/reactions endpoint.
This refactors add_reaction in python_examples.py to use the
openapi_test_function decorator and validate result with
validate_against_openapi_schema. Minor changes have been made to the
OpenAPI format data for /messages/{message_id}/reactions endpoint.

This also adds add-emoji.md to templates/zerver/api and adds
add-emoji to rest-endpoints.md (templates/zerver/help/include).
2020-03-08 19:04:15 -07:00
akashaviator 9c63976da5 api: Refactor get_members_backend in zerver/views/users.py.
This refactors get_members_backend to return user data of a single
user in the form of a dictionary (earlier being a list with a single
dictionary).

This also refactors it to return the data with an appropriate key
(inside a dictionary), "user" or "members", according to the type of
data being returned.

Tweaked by tabbott to use somewhat less opaque code and simple OpenAPI
descriptions.
2020-03-08 18:43:30 -07:00
Tim Abbott 2c75b39078 templates: Delete show_debug feature.
As far as I know, this hasn't been used in at least 5 years, and I'm
not sure there's a real use case for it with the current app.
2020-03-08 18:34:59 -07:00
Tim Abbott ccf63ac66b decorators: Restructure get_client_name interface.
Previously, get_client_name was responsible for both parsing the
User-Agent data as well as handling the override behavior that we want
to use "website" rather than "Mozilla" as the key for the Client object.

Now, it's just responsible for User-Agent, and the override behavior
is entirely within process_client (the function concerned with Client
objects).

This has the side effect of changing what `Client` object we'll use
for HTTP requests to /json/ endpoints that set the `client` attribute.
I think that's in line with our intent -- we only have a use case for
API clients overriding the User-Agent parsing (that feature is a
workaround for situations where the third party may not control HTTP
headers but does control the HTTP request payload).

This loses test coverage on the `request.GET['client']` code path; I
disable that for now since we don't have a real use for that behavior.

(We may want to change that logic to have Client recognize individual
browsers; doing so requires first using a better User-Agent parsing
library).

Part of #14067.
2020-03-08 14:19:50 -07:00
Tim Abbott 53cc00c21c messages: Ban the sender property when not mirroring.
The "sender" property in `send_message_backend` is meant to only do
something when doing Zephyr mirroring (or similar).  We should help
clients behave correctly by banning this property in requests that are
not specifically requesting mirroring behavior.

This commit requires changes to a number of tests that incorrectly
passed this parameter or didn't use the right setup for mirroring.
2020-03-08 14:09:32 -07:00
Tim Abbott cf897cc4b6 test_messages: Convert Zephyr mirror tests to use API.
The special Zephyr mirroring logic is only intended to be used via the
API, so this sets up a more effective test.  It also allows us to
remove certain Client parsing logic for the /json/ views using session
authentication.
2020-03-08 13:38:20 -07:00
sahil839 6a791e1939 subscriber list: Fix display of email addresses in subscribers list.
Original email address is shown to admin users in subscriber list when
email_address_visibilty is set to "Admins only" by passing delivery_email
at required places.  Email address are not shown to non-admin users when
visibility is set to "Admins only".

Tweaked by tabbott to fix a few bugs and dead code.

Fixes a part of #13541.
2020-03-08 12:42:08 -07:00
Tim Abbott d0927d38fc streams: Use user IDs in membership list to track subscribers.
User IDs are more robust than email addresses as they don't change
with time, and also don't have complications with
different email_address_visibility settings.
2020-03-08 12:37:48 -07:00
Mateusz Mandera fe0f381914 populate_db: Don't restrict email domains by default in tests and dev.
The email domain restriction to @zulip.com is annoying in development
environment when trying to test sign up. For consistency, it's best to
have tests use the same default, and the tests that require domain
restriction can be adjusted to set that configuration up for themselves
explicitly.
2020-03-07 18:38:59 -08:00
Tejas Tank b4e8de45a9
portico: Hide download buttons on initial render of `/apps` page.
This will hide all the download buttons on the initial render of the `/apps`
page. It'll hide them until the JavaScript is loaded and calls `update_page`
method to render appropriate button.

We are not using JS to hide the buttons as it still will result in displaying the
buttons and not hide them until JS kicks in. Optimal solution is to set them as
hidden HTML elements and let jQuery override it's display attribute later.

Fixes #14134.
2020-03-07 18:33:31 -08:00
Tim Abbott 5835023021 tests: Use user IDs internally in send message helpers.
This uses the better, modern, user ID based API for sending messages
internally in the test suite, something that's convenient to do as a
follow-up to the migration to pass UserProfile objects to these
functions.
2020-03-07 18:31:13 -08:00
Steve Howell 5e2a32c936 tests: Use users in send_*_message.
This commit mostly makes our tests less
noisy, since emails are no longer an important
detail of sending messages (they're not even
really used in the API).

It also sets us up to have more scrutiny
on delivery_email/email in the future
for things that actually matter.  (This is
a prep commit for something along those
lines, kind of hard to explain the full
plan.)
2020-03-07 18:30:13 -08:00
Tim Abbott 35b444d59c api docs: Document historical changes to typing API.
Along with other recent changes, this fixes #13286.
2020-03-06 17:49:53 -08:00
Divyanshu Agrawal 043b55b5af user status: Save status on enter keypress.
This is a common UX pattern for forms - a user would expect the
input to be submitted on hitting enter.

So, create a 'keypress' event listener on the input field for the
new status, which calls 'submit_new_status' on enter key press.
2020-03-06 17:13:50 -08:00
Wbert Adrian Castro Vera 40a6602b09 filters: Refactor filter_with_new_topic to expand its functionality.
This intent is that we'll be able to reuse this when editing streams
as well.

* Rename method: filter_with_new_topic to filter_with_new_param.
* Fix tests and method calls.
2020-03-06 17:10:26 -08:00
Vishnu KS 1c6435d4cc validator: Optionally record a type_structure attribute.
We plan to use these records to check and record the schema of Zulip's
events for the purposes of API documentation.

Based on an original messier commit by tabbott.

In theory, a nicer version of this would be able to work directly off
the mypy type system, but this will be good enough for our use case.
2020-03-06 17:07:14 -08:00
Tim Abbott 9230213bde settings: Add EMAIL_ADDRESS_VISIBILITY_NOBODY.
This extends our email address visibility settings to deny access to
user email addresses even to organization administrators.

At the moment, they can of course change the setting (which leaves an
audit trail), but in the future only organization owners will be able
to change that setting.

While we're at this, we rewrite the settings_data.js test to cover all
the cases in a more consistent way.

Fixes #14111.
2020-03-06 16:34:08 -08:00
Tim Abbott 914cda9e2d test_classes: Fix api credentials with email_address_visibility setting.
This isn't the only bug in our testing libraries with
EMAIL_ADDRESS_VISIBILITY; but we don't have a lot of tests that need
to deal with that set of settings.
2020-03-06 16:33:16 -08:00
Mateusz Mandera 64b85415f5 migrations: Fix unused import error. 2020-03-06 12:17:19 -08:00
Siddharth Varshney 2a3038797b portico: Update google play and app store badges.
This updates update the download android and ios app button on
/apps/android and /apps/ios routes respectively to use the official
badges provided by the google and apple.

We also clean up some of the JavaScript implementing the page.

Fixes #14061.
2020-03-06 12:12:02 -08:00
Steve Howell 1b4cac6734 models: Cache failures to find user in get_user_by_api_key.
We will cache failed lookups with None.  The
use case here is that broken API clients may
continually ask for the same wrong API key, and
we want to handle that as quickly as possible.
2020-03-06 12:02:02 -08:00
Divyanshu Agrawal 8b2b693231 settings page: Apply dark theme to 'code' elements in night mode.
The example regexes for linkifier settings are not themed according
to the user's dark theme setting.

Modify 'night_mode.scss' to render all 'code' elements with dark
theme.
2020-03-06 11:59:28 -08:00
Steve Howell f2b8eef21a refactor: Avoid hacky use of ValidationError.code.
We were using `code` to pass around messages.

The `code` field is designed to be a code, not
a human-readable message.

It's possible that we don't actually need two
flavors of messages for these type of validations,
but I didn't want to change that yet.

We **definitely** don't need to put two types of
message in the exception, so I fix that.  Instead,
I just have the caller ask what level of detail
it needs.

I added a non-verbose message for the case of
system bots.

I removed the non-translated version of the message
for deactivated accounts, which didn't have test
coverage and is slightly more prone to leaking
email info that we don't want to leak.
2020-03-06 11:53:22 -08:00
Steve Howell 62fb3ad801 refactor: Move validate_email_not_already_in_realm.
We move this to email_validation.py.
2020-03-06 11:53:22 -08:00
Steve Howell 7e55cab429 invite performance: Reduce queries to find existing users.
In the prep commits leading up to this, we split
out two new helpers:

    validate_email_is_valid
    get_errors_for_new_emails

Now when we validate invites we use two separate
loops to filter our emails.

Note that the two extracted functions map to two
of the data structures that used to be handled
in a single loop, and now we break them out:

    errors = validate_email_is_valid
    skipped = get_errors_for_new_emails

The first loop checks that emails are even valid
to begin with.

The second loop finds out whether emails are already
in use.

The second loop takes advantage of this helper:

    get_errors_for_new_emails

The second helper can query all potential new emails
with a single round trip to the database.

This reduces our query count.
2020-03-06 11:53:22 -08:00
Steve Howell 220c2a5ff3 performance: Add get_users_by_delivery_email().
The main purpose of this new function is to allow
us to validate emails in bulk, which we don't do
yet (still setting the stage for that).

This is still a speedup, though, since in our
caller we grab only three fields now.

And other than that, we're essentially doing
the same query for the single-email case, just
outside the loop.
2020-03-06 11:53:22 -08:00
Steve Howell b35ffde5fb tests: Avoid calling actions.validate_email().
We are trying to kill off `validate_email`, so
we no longer call it from these tests.

These tests are already kind of low-level in
nature, so testing the more specific helpers
here should be fine.

Note that we also make the third parameter
to `validate_email` non-optional in this commit,
to preserve 100% coverage.  This is really just
refactoring noise--we will soon eliminate the
entire function, but I didn't want to do everything
in a huge commit.
2020-03-06 11:53:22 -08:00
Steve Howell 6f62c993a6 refactor: Extract get_existing_user_errors.
This is a prep commit that will allow us
to more efficiently validate a bunch of
emails in the invite UI.

This commit does not yet change any
behavior or performance.

A secondary goal of this commit is to
prepare us to eliminate some hackiness
related to how we construct
`ValidationError` exceptions.

It preserves some quirks of the prior
implementation:

   - the strings we decided to translate
     here appear haphazard (and often
     get ignored anyway)

   - we use `msg` in most codepaths,
     but use `code` for invites

Right now we never actually call this with
more than one email, but that will change
soon.

Note that part of the rationale for the inner
method here is to avoid a test coverage bug
with `continue` in loops.
2020-03-06 11:53:22 -08:00
Steve Howell ad85e286de user settings: Inline call to validate_email.
We are trying to elminate the version of
`validate_email` that lives in `actions.py`.

Inlining it barely increases the code size, and
it removes some noise related the three-item
tuple that `check_incoming_email` returns.
2020-03-06 11:53:22 -08:00
Steve Howell 689aca9140 refactor: Extract validate_email_is_valid().
This has two goals:

    - sets up a future commit to bulk-validate
      emails

    - the extracted function is more simple,
      since it just has errors, and no codes
      or deactivated flags

This commit leaves us in a somewhat funny
intermediate state where we have
`action.validate_email` being a glorified
two-line function with strange parameters,
but subsequent commits will clean this up:

    - we will eliminate validate_email
    - we will move most of the guts of its
      other callee to lib/email_validation.py

To be clear, the code is correct here, just
kinda in an ugly, temporarily-disorganized
intermediate state.
2020-03-06 11:53:22 -08:00
Steve Howell 4f5b07a7e6 refactor: Extract zerver/lib/email_validation.py. 2020-03-06 11:53:22 -08:00
Steve Howell 30b43605c3 invite performance: Reduce RealmDomain queries.
We now use the `get_realm_email_validator()`
helper to build an email validator outside
the loop of emails in our invite list.

This allows us to perform RealmDomain queries
only once per request, instead of once per
email.
2020-03-06 11:53:22 -08:00
Steve Howell ce8f6797c7 performance: Optimize get_realm_email_validator.
We now query RealmDomain objects up front.  This
change is minor in most circumstances--it sometimes
saves a round trip to the database; other times,
it actually brings back slightly more data
(optimistically).

The big win will come in a subsequent commit,
where we avoid running these queries in a loop
for every callback.

Note that I'm not sure if we intentionally
omitted checks for emails with "+" in them
for some circumstances, but I just preserved
the behavior.
2020-03-06 11:53:22 -08:00
Steve Howell ddbc536739 refactor: Extract get_realm_email_validator.
This change sets us up to use the same realm
data for multiple email validations.
2020-03-06 11:53:22 -08:00
Steve Howell 57f1aa722c refactor: Rename validate_email_for_realm.
Now called:

    validate_email_not_already_in_realm

We have a separate validation function that
makes sure that the email fits into a realm's
domain scheme, and we want to avoid naming
confusion here.
2020-03-06 11:53:22 -08:00