Commit Graph

11543 Commits

Author SHA1 Message Date
Anders Kaseorg 04919528e7 models: Parameterize .extra(where=["… IN …"]) with tuple/list adapter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg de22f7a378 migrations: Use cursor.execute correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 859c8b169e stream_recipient: Use cursor.execute correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 68e78c809f get_recent_private_conversations: Use cursor.execute correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 3aab9c03a9 fix_unreads: Use cursor.execute correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 14bbfe6ffb bulk_get_subscriber_user_ids: Use cursor.execute correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 2604ebba38 import_message_data: Use psycopg2.extras.execute_values.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 6649c8e35f bulk_insert_ums: Use psycopg2.extras.execute_values.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Dinesh dc90d54b08 auth: Add Sign in with Apple support.
This implementation overrides some of PSA's internal backend
functions to handle `state` value with redis as the standard
way doesn't work because of apple sending required details
in the form of POST request.

Includes a mixin test class that'll be useful for testing
Native auth flow.

Thanks to Mateusz Mandera for the idea of using redis and
other important work on this.

Documentation rewritten by tabbott.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-09 17:29:35 -07:00
David Wood f0f42f7a94 integrations: Add Thinkst Canary integration.
This commit adds an integration for Thinkst Canaries - physical, VM and
cloud-based canaries for detecting attackers to a network. Thinkst
Canaries can send webhook alerts when canaries have been tripped, and
this integration will post Zulip messages when these webhooks are
received.

Signed-off-by: David Wood <david@davidtw.co>
2020-06-09 16:28:28 -07:00
Tim Abbott e242ddc848 page_params: Add community_topic_editing_limit_seconds.
This was previously hardcoded with agreement between the Zulip backend
and frontend as 86400 seconds (1 day).  Now, it's still hardcoded in
the backend, but arranged in a way where we could add a setting
without any changes to the mobile and terminal apps to update logic.

Fixes #15278.
2020-06-09 14:40:12 -07:00
Tim Abbott cb8290ac57 test_home: Stop checking for removed string. 2020-06-08 23:09:21 -07:00
Tim Abbott 2beaf2cab2 pointer: Remove have_initial_messages code.
This was implemented in 2012 to avoid showing a loading indicator for
fetching messages for users with no message history.  However, the
Zulip onboarding UI always creates some message history, and fetching
history is fast, so this is likely clutter more than a useful
optimization.
2020-06-08 22:36:18 -07:00
Anders Kaseorg 47b4e45931 markdown_test_cases: Update encoded zulipchat.com links too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 19:47:07 -07:00
Tim Abbott 7b8ba5ebd9 docs: Update most remaining references to zulipchat.com.
In some cases, the cleanest tweak is to replace references to the
domain with Zulip Cloud, the product.
2020-06-08 18:10:45 -07:00
Tim Abbott 95be7dcbab portico: Remove mystery hunt landing page.
This was written by Rishi for a very brief purpose a few years ago,
and it doesn't serve much purpose now other than to be a place we
update in code sweeps.
2020-06-08 18:10:45 -07:00
Tim Abbott 71078adc50 docs: Update URLs to use https://zulip.com.
We're migrating to using the cleaner zulip.com domain, which involves
changing all of our links from ReadTheDocs and other places to point
to the cleaner URL.
2020-06-08 18:10:45 -07:00
Dinesh d30f11888a logging: Set up a different logger for each backend.
Adds a top-level logger in `settings.LOGGING` `zulip.auth`
with the default handlers `DEFAULT_ZULIP_HANDLERS` and
an extra hanlder that writes to `/var/log/zulip/auth.log`.

Each auth backend uses it's own logger, `self.logger` which
is in form 'zulip.auth.<backend name>'.

This way it's clear which auth backend generated the log
and is easier to look for all authentication logs in one file.

Besides the above mentioned changes, `name` attribute is added to
`ZulipAuthMixin` so that these logging kind of calls wouldn't raise
any issues when logging is tried in a class without `name` attribute.

Also in the tests we use a new way to check if logger calls are made
i.e. we use `assertLogs` to test if something is logged.

Thanks to Mateusz Mandera for the idea of having a seperate logger
for auth backends and suggestion of using `assertLogs`.
2020-06-08 17:42:07 -07:00
Anders Kaseorg b1b34b6f6a dev_login: Move owners to the top.
Since owners have the highest privilege level, it made little sense to
sandwich them between administrators and guests.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 17:26:34 -07:00
sahil839 7de23b8b5c users: Remove short_name and client_id from get_profile_backend.
This commit removes short_name and client_id fields from the user
objects returned by get_profile_backend because neither of them
had a purpose.

* short_name hasn't been present anywhere else in the Zulip API for
  several years, and isn't set through any coherent algorithm.
* client_id was a forgotten 2013-era predecessor to the queue_id field
  returned by the register_event_queue process.

The combination of these changes gets us close to having `get_profile`
have the exact same format as other endpoints fetching a user object.
2020-06-08 17:01:08 -07:00
sahil839 1f5778bad7 users: Refactor get_profile_backend to be based on format_user_row.
This commit changes get_profile_backend to be based on format_user_row
such that it's a superset of the fields for our other endpoints for
getting data on a user.

To be clear, this does not removes any of the exisiting fields, that
were returned by this endpoint.

This change adds some fields to the User object returned by the
endpoint. API docs are updated accordingly for the added fields.
2020-06-08 16:57:44 -07:00
sahil839 2f7d684a84 slack_import: Map slack owners to zulip realm owners.
Slack owners and primary owners will be mapped to zulip
realm owners on import.

Previously, we mapped the owner and primary owner roles of slack
to realm admins in zulip. As we have added ROLE_REALM_OWNER in
8bbc074, we now map slack owners and primary owners to owners in
zulip.

Tests are modified for checking all the 3 cases-
 - Slack workspace primary owner
 - Slack workspace owner
 - Slack workspace admin

This commit also has docs changes in 'import-from-slack.md'.
2020-06-08 16:22:54 -07:00
Anders Kaseorg 8e4f22c184 auth: Require algorithms setting for JWT auth.
Calling jwt.decode without an algorithms list raises a
DeprecationWarning.  This is for protecting against
symmetric/asymmetric key confusion attacks.

This is a backwards-incompatible configuration change.

Fixes #15207.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 16:22:25 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg 8c15081069 python: Further pyupgrade changes.
Generated by pyupgrade --py3-plus --keep-percent-format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:23:10 -07:00
Anders Kaseorg 444fbbf964 python: Whitespace fixes from autopep8.
Generated by autopep8.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:21:30 -07:00
Anders Kaseorg 549fbeac72 test_narrow: Convert a triple-quoted format string to single-quoted.
This works around a semgrep bug following the conversion to f-strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:19:59 -07:00
Anders Kaseorg 139cb8026f auth: Accept next as POST parameter in POST requests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 11:07:32 -07:00
Johannes Hochreiter d78d9099d1 gitlab: added pipeline number to event notification
- link to pipeline now also features the pipeline number
- updated tests accordingly
2020-06-07 14:01:04 -07:00
Johannes Hochreiter 60077f3097 gitlab: Added artifact link to pipeline event body
- if artifact was built file name, download link and browse link will be added to pipeline event body
- added test for pipeline with built artifacts
2020-06-07 14:01:04 -07:00
Aman Agrawal e2ed155310 do_update_message: Send old topic with stream edit event.
Old topic of the msg edit event can be used to help the client
calculate useful information such as if a change
in current narrow is required.

This fixes our re narrow logic after a stream edit of a topic, with
no change in topic name itself, since the original topic was not
present in the event received and hence the `orig_topic` was
undefined in this case.
2020-06-07 11:20:34 -07:00
sahil839 c394337118 tests: Refactor 'attempt_unsubscribe_of_principal` to accept list of users.
This adds 'target_users' parameter to 'attempt_unsubscribe_of_principal`
function in test_subs.py, which accepts list of UserProfile objects to be
unsubscribed, instead of defining users in the function itself.

This change makes the code cleaner and more readable.

Also, 'other_user_subbed' parameter is changed to 'target_users_subbed'
to clearly depict the use of this parameter.
2020-06-07 11:18:37 -07:00
Anders Kaseorg 19cc22e5ab queue: Fix types to reflect that Pika channels transmit bytes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-07 11:09:24 -07:00
anshgit101 7f4aad4511 populate_db:Add uppercase topic name randomly. 2020-06-06 09:25:13 -04:00
Anders Kaseorg a236256d64 test_slack_importer: Fix mocked_requests_get type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 20:05:18 -07:00
Anders Kaseorg 4b44d87477 test_push_notifications: Fix mock_apns type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 20:05:18 -07:00
Anders Kaseorg 22178c169e request: Replace type argument hack with a different workaround.
This works in mypy 0.770 and is needed for mypy 0.780.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 20:05:18 -07:00
Tim Abbott 42f1cb3444 management: Fix delete_old_unclaimed_attachments. 2020-06-05 17:06:19 -07:00
Kartik Srivastava 5cb1edcadc openapi/python_examples: Update examples for add/remove subscriptions.
This updates the examples to use user ids to add or remove subscriptions
from a stream.
2020-06-05 16:11:04 -07:00
sahil839 43bff1f67a streams: Extract removing_someone_else logic to a function.
This commit extracts removing_someone_else logic to the function
'check_if_removing_someone_else' for better readability.

This is a follow up to 2187c84.
2020-06-05 16:04:28 -07:00
Aman Agrawal a096f34cab move_topic_to_stream: Add option to disable breadcrumb messages.
Option to disable breadcrumb messages were given in both message edit
form and topic edit stream popover.

User now has the option to select which stream to send the notification
of stream edit of a topic via checkboxes in the UI.
2020-06-05 12:28:51 -07:00
Aman Agrawal ec257f6ec3 to_dict_uncached: Pipe realm_id from the caller functions.
We pipe realm_id through functions where it is available,
this helps us avoid doing query for realm_id in loop when
multiple messages are being processed.
2020-06-05 12:03:31 -07:00
Aman Agrawal 4b6ce8473e test_messages: Check number of queries when a topic is moved. 2020-06-05 12:03:31 -07:00
Mateusz Mandera 5b746b4128 docs: Add missing step to dropbox webhook instructions. 2020-06-05 10:21:25 -07:00
Anders Kaseorg 0bcd4a6281 email_mirror: Avoid parameters that shadow names in their own types.
They confuse semgrep, and also, like, people in general.

https://github.com/returntocorp/semgrep/issues/923

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:35:11 -07:00
Anders Kaseorg 8d20d1e632 models: Set a timezone on the MutedTopic.date_muted default.
Fixes warnings like this:

/srv/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField MutedTopic.date_muted received a naive datetime (2020-01-01 00:00:00) while time zone support is active.
 warnings.warn("DateTimeField %s received a naive datetime (%s)"

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:34:17 -07:00
Anders Kaseorg 1f565a9f41 timezone: Use standard library datetime.timezone.utc consistently.
datetime.timezone is available in Python ≥ 3.2.  This also lets us
remove a pytz dependency from the PostgreSQL scripts.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:34:17 -07:00
Anders Kaseorg 0797626911 bugdown: Avoid deprecated ElementTree.getchildren method.
Fixes warnings like these with python -Wd:

/home/circleci/zulip/zerver/lib/bugdown/__init__.py:327: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  for child in currElementPair.value.getchildren():
/home/circleci/zulip/zerver/lib/bugdown/__init__.py:328: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  if child.getchildren():
/home/circleci/zulip/zerver/lib/bugdown/__init__.py:282: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  for child in currElement.getchildren():
/home/circleci/zulip/zerver/lib/bugdown/__init__.py:283: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  if child.getchildren():

https://docs.python.org/3.8/library/xml.etree.elementtree.html#xml.etree.ElementTree.Element.getchildren

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:17:08 -07:00
Anders Kaseorg b94b350ff0 bot_config: Avoid deprecated ConfigParser.readfp method.
Fixes this warning with python -Wd:

/home/circleci/zulip/zerver/lib/bot_config.py:69: DeprecationWarning: This method will be removed in future versions.  Use 'parser.read_file()' instead.
  config.readfp(conf)

https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.readfp

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:16:17 -07:00
Anders Kaseorg 95f29b5409 tests: Avoid deprecated TestCase method aliases.
Fixes these warnings with python -Wd:

/home/circleci/zulip/zerver/tests/test_middleware.py:43: DeprecationWarning: Please use assertRegex instead.
  self.assertRegexpMatches(
/home/circleci/zulip/zerver/webhooks/travis/tests.py:40: DeprecationWarning: Please use assertNotEqual instead.
  self.assertNotEquals(msg.topic_name(), self.TOPIC)

https://docs.python.org/3/library/unittest.html#deprecated-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:15:52 -07:00