Commit Graph

3953 Commits

Author SHA1 Message Date
okmanl 2a1305de9f lint: Add a rule to avoid msgid as a Python variable name.
This is for consistency with our usual patterns, see #12995.  We will
need a similar commit for JavaScript to complete #12995.
2019-08-17 12:47:13 -07:00
Wyatt Hoodes dc191b0be4 test_queue_worker: Test actual code path for slow_queries.
Instead of just mocking some fake events, we use the code
path that generates slow query events and publishes them
to SlowQueryWorker.

This test improvement would have got a recent potential regression
caught in code review.
2019-08-17 12:38:02 -07:00
Tim Abbott 26ac3ebd3e openapi: Suggest the -sS options to curl.
These options prevent curl from doing the downloading progress bar
(which is clutter).
2019-08-17 11:35:08 -07:00
Tim Abbott 2366490ffc openapi: Pass api_url to curl example generation.
Our new curl example generation logic was broken, in that it hardcoded
localhost:9991 (without an HTTP method or anything) as the API URL.

It requires a bit of plumbing to make this possible.
2019-08-17 11:35:08 -07:00
Priyank Patel 3680393b47 messages: Support passing user ID for stream operator.
ok_to_include_history fuction was updated to expect stream ID.

Fixes part of #9474.
2019-08-17 11:19:12 -07:00
Mateusz Mandera cb2c9b04b3 generic_bulk_cached_fetch: Only call query_function if necessary. 2019-08-15 17:14:02 -07:00
Mateusz Mandera 898bc52538 models: Add bulk_get_huddle_user_ids function. 2019-08-15 17:03:17 -07:00
Tim Abbott 2ada0a9bad management: Fix password reset emails being sent to deactivated users.
Apparently, the filters written for the send_password_reset_email (and
some other management commands) didn't correctly consider the case of
deactivated users.

While some commands, like syncing LDAP data (which can include whether
a user should be deactivated) want to process all users, other
commands generally only want to interact with active users.  We fix
this and add some tests.
2019-08-14 10:50:45 -07:00
Wyatt Hoodes 001ff72861 data exports: Add upload and message limit test. 2019-08-12 19:52:46 -07:00
Wyatt Hoodes 896886b412 realm_export: Add endpoint for export deletion. 2019-08-12 17:51:46 -07:00
Wyatt Hoodes 777dcdd272 test_realm_export: Clean up export testing.
* Added comments.
* Refactored `export_object` to `audit_log_entry`.
* Cleaned up white space.
* Removed uses of `getattr`.
2019-08-12 17:51:46 -07:00
Wyatt Hoodes 7c118096e7 test_events: Remove print statement from self.client_post.
Leftover from prior debugging.
2019-08-12 17:51:46 -07:00
Wyatt Hoodes 7d178bbb0f queue_processors: Clean up the extra_data dict code.
We don't want to add a `deleted_timestamp` key until
the export is actually deleted.
2019-08-12 17:51:46 -07:00
Wyatt Hoodes 6f6efa516d exports: Refactor extra_data to export_data. 2019-08-12 17:51:46 -07:00
Wyatt Hoodes 7a2a1f29ad exports: Refactor event_time to export_time timestamp.
The time of the event was incorrectly being sent
as a datetime object.
2019-08-12 17:51:46 -07:00
Anders Kaseorg e0cf3d0e76 zerver: Accept HEAD requests wherever GET requests are accepted.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-12 16:47:41 -07:00
Hemanth V. Alluri d425ab5ae5 test_templates: Add a verbose error messages for rendering failures.
If a template fails to render in test_templates because of a missing
context variable, we should better explain the reason for the failure.
2019-08-12 16:24:08 -07:00
Anders Kaseorg 72655611ce requirements: Use maintained fork django-sendfile2 of django-sendfile
The original seems to be unmaintained
(johnsensible/django-sendfile#65).  Notably, this fixes a bug in the
filename parameter, which perviously showed the Python 3 repr of a
byte string (johnsensible/django-sendfile#49).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-12 15:40:08 -07:00
Anders Kaseorg 7cd11d0559 test_users: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Anders Kaseorg 75a01810e3 test_outgoing_webhook_interfaces: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Anders Kaseorg 1ffbc0009d test_openapi: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Anders Kaseorg 9d33200fb2 test_messages: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Anders Kaseorg 4671e4ca5c test_i18n: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Anders Kaseorg c453c4b602 test_events: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:42:33 -07:00
Priyank Patel 72bf7e2181 test_narrow: Rename stream_id -> stream_recipient_id.
It looks like this isn't actually a stream id, doing
get_stream_by_id_in_realm(stream_id, stream_messages[0].get_realm())
throws error.
2019-08-09 17:31:53 -07:00
meiomorphism 1af72a2745 subscriptions: Avoid sending unneeded subscriber information.
The `users/me/subscriptions` endpoint accidentally started returning
subscriber information for each stream.  This is convenient, but
unnecessarily costly for those clients which either don't need it
(most API apps) or already acquire this information via /register
(including Zulip's apps).

This change removes that data set from the default response.  Clients
which had come to rely on it, or would like to rely on it in future,
may still access it via an additional documented API parameter.

Fixes #12917.
2019-08-09 16:03:21 -07:00
Alexandra Ciobica e5e45c9a25 auth: Change page title and add description for the list.
I changed the class of the title in order to use the same styling as the
 other similar pages (like `/accounts/go` or `/login`).

Changed the related test.
2019-08-08 11:12:51 -07:00
Alexandra Ciobica f109dcce9c auth: Add logic for avatars to the GitHub auth email selection page.
For the emails that are associated to an existing account in an
organisation, the avatars will be displayed in the email selection
page.  This includes avatar data in what is passed to the page.

Added `avatar_urls` to the context in `test_templates.py`.
2019-08-08 11:12:51 -07:00
Alexandra Ciobica d4ccd73ae3 auth: Remove `@users.noreply.github.com` from the email selection list.
Apparently GitHub changed the email address for these; we need to
update our code accordingly.

One cannot receive emails on the username@users.noreply.github.com, so
if someone tries creating an account with this email address, that
person would not be able to verify the account.
2019-08-08 11:12:51 -07:00
Anders Kaseorg 7e92ff9d0a request: Delete request.pyi and make request.py type check.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-07 15:37:21 -07:00
Tim Abbott 4e631876f4 test_events: Fix realm export test to actually work.
The previous iteration still had the failure mode of not actually
testing anything, because it didn't trigger the data export code path
(and in fact was getting an HTTP 401 authentication denied error).
2019-08-07 14:18:27 -07:00
Wyatt Hoodes 22842dab34 events: Rename notify_export_completed.
notify_realm_export is more reasonable for the context of doing
deletion events as well.
2019-08-07 14:18:27 -07:00
Wyatt Hoodes 7853e9d2de test_events: Fix broken test case.
This test was broken due to using an empty `RealmAuditLog`
table.  We fix this by mocking the creation of an export,
thus creating an entry, similar to what we do in our other
tests.
2019-08-07 12:05:44 -07:00
Wyatt Hoodes f93efcb89c test_helpers: Refactor dummy file creation for broader use. 2019-08-07 12:05:44 -07:00
Wyatt Hoodes 11db0c23fb exports: Update extra_data field to a JSON structure.
We add the `deleted_timestamp` key to the new `extra_data`
dictionary.
2019-08-07 12:04:28 -07:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
neiljp (Neil Pilgrim) 5ab64daecc mypy: Remove type ignore by defining ProfileDataElement using TypedDict. 2019-08-06 23:24:56 -07:00
Hemanth V. Alluri ac69717604 openapi: Migrate /messages to use the new cURL example system.
Now that we can create cURL examples based on the OpenAPI
documentation. We can begin using simple one line tags in
the documentation instead of manually creating cURL examples.

Fixes part of #12878.
2019-08-05 21:27:24 -07:00
Hemanth V. Alluri 5af753d940 api_code_examples: Parse kwargs/mods for example code generation.
Now we can also include extra keyword arguments to specify
modifications in how the example code should be generated
in the generate_code_example template tag.

E.g. generate_code_example(curl, exclude=["param1", "param2"])
2019-08-05 21:27:23 -07:00
Hemanth V. Alluri b20cf095e7 curl: Add code to auto generate cURL examples from OpenAPI docs.
This commit extends api_code_examples.py to support automatically
generating cURL examples from the OpenAPI documentation. This way
work won't have to be repeated and we can also drastically reduce
the chance of introducing faulty cURL examples (via. an automated
test which can now be easily created).
2019-08-05 21:13:14 -07:00
Hemanth V. Alluri 0682f4f12a openapi: Extend type checking to check the elements of lists.
Now that the type of the elements of each array/list are also
specified in the OpenAPI documentation, we can extend our type
checking even further.
2019-08-05 21:06:19 -07:00
Hemanth V. Alluri 14c0a387cf openapi: Remove /users/me from pending endpoints.
This commit progress our efforts to reduce pending_endpoints
as well as to migrate away from templates/zerver/api/fixtures
and towards our OpenAPI documentation.

Similar to commit d62b75fc.
2019-08-05 21:06:19 -07:00
Rishi Gupta b37c6a8644 custom profile: Update a few error messages. 2019-08-05 20:59:47 -07:00
Rishi Gupta 8167b5b645 email: Remove special integration doc behavior when gateway not set.
The current code looks like it's trying to redirect /integrations/doc/email
to /integrations when EMAIL_GATEWAY_PATTERN is not set.

I think it doesn't currently do this. The test for that pathway has a bug:
self.get_doc('integrations/doc-html/email', subdomain='zulip') needs a
leading slash, and putting the slash back in results in the test failing.

This redirection is not really desired behavior -- better is to
unconditionally show that the email integration exists, and just point the
user to https://zulip.readthedocs.io/en/latest/production/email-gateway.html
(this is done in a child commit).
2019-08-05 17:33:05 -07:00
Anders Kaseorg 68dd8e4ec8 mypy: Migrate from mypy_extensions to typing_extensions.
This gives us access to typing_extensions.Deque, which was not added
to typing until 3.5.4.

(PROVISION_VERSION is not bumped because the transitive dependency set
in dev.txt hasn’t changed.)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-05 17:24:09 -07:00
Anders Kaseorg 86a7fdddd7 events: Check last_event_id for validity, take 2.
This verifies that the client passed a last_event_id that actually
came from the queue instead of making up an ID from the future.  It
turns out one of our tests was making up such an ID, but legitimate
clients are expected not to do so.

The previous version of this commit (commit
e00d4be6d5, #12888) had to be reverted
(commit b86c5cc490) because it was
missing the `to_dict`/`from_dict` migration code.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-05 17:18:49 -07:00
Rohitt Vashishtha a7f2bedb15 markdown: Enable hashheadings syntax.
Our implementation requires at least 1 space after the
'#' not not break existing linkifiers like '#123', etc.
that generally follow the convention we show in linkifier
examples.

- [valid]  : # Hello
- [valid]  : #  Hello
- [invalid]: #Hello

For the frontend, we have taken the code from v0.7.0 of
upstream marked and made minor changes to avoid having
to refactor a significant part of our marked code.

For the backend, we merely have to change the regex to
force require spaces after #, and add hashheader to our
list of blockparsers.

Fixes #11418.
2019-08-02 15:15:34 -07:00
Tim Abbott b86c5cc490 Revert "events: Check last_event_id for validity."
This isn't correct without a proper migration for existing queues,
which may not be implementable.

This reverts commit e00d4be6d5.
2019-08-02 14:44:35 -07:00
Tim Abbott 2738e909f8 event_queue: Expand testing of event queue save/restore.
This verifies that every valid state of our event queue system will be
properly save/restored via the from_dict methods.
2019-08-02 14:43:47 -07:00
Tim Abbott 3a6e5dad53 tests: Move EventQueueTest to test_event_queue.py.
This seems more appropriate, especially as we expand this library.
2019-08-02 14:43:12 -07:00