Commit Graph

961 Commits

Author SHA1 Message Date
Anders Kaseorg 4bfffc9f74 docs: Fix list item indentation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 672358c5da docs: Migrate from recommonmark to MyST-Parser.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.

https://myst-parser.readthedocs.io/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 1bdb7b1141 mypy: Add boto3-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 05124737f7 mypy: Add backoff-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 09564e95ac mypy: Add types-psycopg2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 5c90522e69 mypy: Add types-Pillow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 6289803368 mypy: Add types-oauthlib.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 4c73926ad9 mypy: Add types-jsonschema.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 58cfc152a0 mypy: Add types-commonmark.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 1a3e3f9072 mypy: Add lxml-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-04 11:29:20 -07:00
Anders Kaseorg 49907173b2 mypy: Add types-Pygments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 22:31:46 -07:00
Anders Kaseorg 04feadd917 mypy: Add pika-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 22:31:46 -07:00
Priyansh Garg c1885c1b98 requirements: Add pymongo to support reading bson files.
Required for reading bson files from mongodb dump in rocketchat
import tool.
2021-07-15 14:28:23 -07:00
Anders Kaseorg 08592c9731 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-06 08:40:32 -07:00
Anders Kaseorg 3853285241 push_notifications: Replace PyAPNs2 with aioapns.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 20:16:50 -07:00
Anders Kaseorg c56440ded0 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 12:23:06 -07:00
Anders Kaseorg 61e1e38a00 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-07 17:57:51 -07:00
Mateusz Mandera e7033fcd41 dependecies: Install python-social-auth with OIDC support. 2021-05-23 13:30:17 -07:00
Anders Kaseorg e6651a1bbd requirements: Remove ijson.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-20 13:12:16 -07:00
Eeshan Garg 69309540c0 pypi: Upgrade zulip/zulip-bots dependencies to version 0.8.0. 2021-05-20 11:06:48 -07:00
Anders Kaseorg 9ba48c4ed3 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 22:42:39 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Mateusz Mandera 1d9fb4f988 django: Upgrade Zulip to Django 3.2 LTS.
This is a straightforward upgrade in terms of changes needed.

Necessary changes were:
- Set `DEFAULT_AUTO_FIELD`
  https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-auto-created-primary-keys

- `The default_app_config application configuration variable is deprecated, due
  to the now automatic AppConfig discovery.`
  https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery

To handle this one, we can remove default_app_config from
zerver/__init__.py because it satisfies what release notes describe in
https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery:
"Most pluggable applications define an AppConfig subclass in an apps.py
submodule. Many define a default_app_config variable pointing to this
class in their __init__.py.  When the apps.py submodule exists and
defines a single AppConfig subclass, Django now uses that configuration
automatically, so you can remove default_app_config."

An important note is that rebuild-test-database needs to be run after
this upgrade in dev environment - if tests are run with test db that was
built on the previous version, they will fail due to a mysterious bug
(?), where changing attributes of a user and .save()ing after logging in
in the test via self.login_user, causes getting logged out - the next
requests via self.client_get etc. are unauthed for some reason,
unless self.login_user is called again. This behavior is no longer
exhibited upon rebuilding the test db - and I can't reproduce it in
production or dev db. So this can likely be reasonably dismissed as some
quirk of the test client system that won't be relevant in the future and
doesn't impact production.
2021-05-03 08:36:22 -07:00
Cyril Pletinckx 9afde790c6 email: Open a single SMTP connection to send email batches.
Previously the outgoing emails were sent over several SMTP
connections through the EmailSendingWorker; establishing a new
connection each time adds notable overhead.

Redefine EmailSendingWorker worker to be a LoopQueueProcessingWorker,
which allows it to handle batches of events. At the same time, persist
the connection across email sending, if possible.

The connection is initialized in the constructor of the worker
in order to keep the same connection throughout the whole process.
The concrete implementation of the consume_batch function is simply
processing each email one at a time until they have all been sent.

In order to reuse the previously implemented decorator to retry
sending failures a new method that meets the decorator's required
arguments is declared inside the EmailSendingWorker class. This
allows to retry the sending process of a particular email inside
the batch if the caught exception leaves this process retriable.

A second retry mechanism is used inside the initialize_connection
function to redo the opening of the connection until it works or
until three attempts failed. For this purpose the backoff module
has been added to the dependencies and a test has been added to
ensure that this retry mechanism works well.

The connection is closed when the stop method is called.

Fixes: #17672.
2021-04-26 17:27:22 -07:00
Anders Kaseorg 36e938bf9f requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 21:47:33 -07:00
Anders Kaseorg 738532ba51 requirements: Remove django-webpack-loader.
It does not seem like an official version supporting Webpack 4 (to say
nothing of 5) will be released any time soon, and we can reimplement
it in very little code.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-06 09:31:35 -07:00
Anders Kaseorg d55dc6f8f1 requirements: Upgrade python-zulip-api from Git.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 16:31:03 -07:00
Anders Kaseorg 4703256c91 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 08:51:16 -07:00
Anders Kaseorg 6364e1b5f3 requirements: Upgrade talon fork to 1.4.8.
https://github.com/mailgun/talon/pull/200

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-18 17:10:18 -07:00
Alex Vandiver ea2e5de54f requirements: Downgrade jedi dependency for ipython compatibility.
ipython < 7.20.0 is incompatible with jedi >= 0.18.0; it fails to
tab-complete in `./manage.py shell`, as described in
ipython/ipython#12740.

We cannot bump the ipython dependency because ipython 7.20.0 requires
Python 3.7, and we must support Python 3.6 due to Ubuntu 18.04
support.  Our only solution is thus to cap the version of `jedi` to
the last one before its API changed.
2021-03-04 18:08:45 -08:00
Alex Vandiver 3207928418 requirements: Fix paths to 'non-"lock" files'; those are the .in files 2021-03-04 18:08:45 -08:00
Alex Vandiver 237dc1eb21 requirements: Update example to match the line above. 2021-03-04 18:08:45 -08:00
Anders Kaseorg 4bbfac8aa9 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Alex Vandiver 12ade4c49d lint: Rely on core gitlint skipping of revert commits.
gitlint 0.13.0 and above skip revert commits[1].  This obviates the
need for a custom rule.

[1] https://jorisroovers.com/gitlint/configuration/#ignore-revert-commits
2021-02-23 14:28:09 -08:00
Anders Kaseorg f21fbea7c7 lint: Replace (most of) pycodestyle with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg f38d68e110 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-10 10:15:50 -08:00
Anders Kaseorg 049d140557 requirements: Upgrade moto to 1.3.17.dev230.
This unblocks the upgrade to responses 0.12.1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-29 12:02:43 -08:00
Anders Kaseorg d33217f2ef requirements: Remove unused httplib2 requirement.
It’s unused since commit 8dd95bd057
(#13718).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg e1815f5d11 requirements: Use Django extra to pull in argon2-cffi.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg 47674c2298 requirements: Use django-two-factor-auth extra to pull in twilio.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg a7bd1f8049 requirements: Upgrade Python requirements. 2021-01-26 13:27:50 -08:00
Mateusz Mandera 1432067959 dependencies: Upgrade to Django 3.1.
https://docs.djangoproject.com/en/3.1/releases/3.1/

- django.contrib.postgres.fields.JSONField is deprecated and should be
  replaced with models.JSONField
-  The internals of the implementation in the postgresql backend have
   changed a bit in
   f48f671223
   and thus we need to make an ugly tweak in test_runner.
- app_directories.Loader.get_dirs() now returns a list of PosixPath so
  we need to make a small tweak in TwoFactorLoader for that (PosixPath
  is not iterable)

Fixes #16010.
2021-01-26 10:20:00 -08:00
Mateusz Mandera bf9e5e52ce dependencies: Upgrade to Django 3.0.
Adjustments made due to changes in Django 3.0:
(https://docs.djangoproject.com/en/3.0/releases/3.0/)

- test_signup: INTERNAL_RESET_URL_TOKEN was moved to
  PasswordResetConfirmView.reset_url_token
- test_message_fetch:
  "add_never_cache_headers() and never_cache() now add the private
  directive to Cache-Control headers."
- "django.utils.html.escape() now uses html.escape() to escape HTML.
  This converts ' to &#x27; instead of the previous equivalent decimal
  code &#39;." - this requires adjusting the expected decimal code
  in some of the string fixtures in tests.
2021-01-26 10:20:00 -08:00
Anders Kaseorg 2ba22d273a requirements: Delete unused pip2.in symlink.
It’s unused since commit a486872a8e
(#14278).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-30 07:54:19 -08:00
Aman Agrawal c685d36821 hipchat_import: Remove tool from codebase.
Remove functions and scripts used by HipChat import tool and
those which will no longer be required in future.
2020-12-23 08:28:49 -08:00
Anders Kaseorg 362d4a6a8d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 14:24:54 -08:00
Anders Kaseorg 13e35bfa94 mypy: Use sqlalchemy-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:17:41 -08:00
Anders Kaseorg 1275613812 requirements: Upgrade mypy to 0.790.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-12 15:44:30 -08:00
Aman Agrawal b2841c7d02 dev_requirements: Upgrade zulint.
This tells users how autofix errors for linters which support it.
This is important since only way to fix prettier errors is
running lint with `--fix` which now the linter will gladly print
with the error.
2020-11-03 14:49:02 -08:00
Anders Kaseorg 7fbd75630f requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-28 23:39:15 -07:00