Commit Graph

2130 Commits

Author SHA1 Message Date
Anders Kaseorg 7c32134fb5 Revert "Revert "request: Refactor to record rate limit data using ZulipRequestNotes.""
This reverts commit 49eab4efef.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-19 16:48:23 -07:00
Tim Abbott b73356e891 portico: Add UCSD case study. 2021-07-18 21:55:57 -07:00
Tim Abbott d876e12157 sentry: Avoid importing rate_limiting at topic level.
This fixes an issue where update-prod-static would crash with the following exception:

2021-07-19 03:59:24,601 upgrade-zulip-stage-2: Building static assets...
Traceback (most recent call last):
  File "./tools/update-prod-static", line 27, in <module>
    os.chdir(settings.DEPLOY_ROOT)
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/conf/__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "./tools/../zproject/settings.py", line 20, in <module>
    from .computed_settings import *  # noqa: F401,F403 isort: skip
  File "./tools/../zproject/computed_settings.py", line 1186, in <module>
    from .sentry import setup_sentry
  File "./tools/../zproject/sentry.py", line 12, in <module>
    from zerver.lib.request import get_request_notes
  File "./tools/../zerver/lib/request.py", line 28, in <module>
    import zerver.lib.rate_limiter as rate_limiter
  File "./tools/../zerver/lib/rate_limiter.py", line 14, in <module>
    from zerver.models import UserProfile
  File "./tools/../zerver/models.py", line 26, in <module>
    from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin, UserManager
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/auth/base_user.py", line 48, in <module>
    class AbstractBaseUser(models.Model):
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/db/models/base.py", line 108, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/apps/registry.py", line 253, in get_containing_app_config
    self.check_apps_ready()
  File "/home/zulip/deployments/2021-07-19-03-58-39/zulip-py3-venv/lib/python3.6/site-packages/django/apps/registry.py", line 136, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Traceback (most recent call last):
  File "/home/zulip/deployments/2021-07-19-03-58-39/scripts/lib/upgrade-zulip-stage-2", line 220, in <module>
    subprocess.check_call(["./tools/update-prod-static"], preexec_fn=su_to_zulip)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./tools/update-prod-static']' returned non-zero exit status 1.
2021-07-18 21:19:11 -07:00
Alya Abbott 844c62c64e portico: Create new "Zulip for communities" page. 2021-07-18 12:10:33 -07:00
Mateusz Mandera 50e9169680 oidc: Add auto_signup configuration option. 2021-07-16 14:40:07 -07:00
Anders Kaseorg fb3ddf50d4 python: Fix mypy no_implicit_reexport errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
PIG208 49eab4efef Revert "request: Refactor to record rate limit data using ZulipRequestNotes."
This reverts commit 3f9a5e1e17.
2021-07-16 09:01:20 -07:00
Alex Vandiver 928dc4bafd sentry: Set environment from `machine.deploy_type` config.
This allows for greater flexibility in values for "environment," and
avoids having to have duplicate definitions of STAGING in
`zproject/config.py` and `zproject/default_settings.py` (due to import
order restrictions).

It does overload the "deploy type" concept somewhat.

Follow-up to #19185.
2021-07-15 15:01:43 -07:00
Alex Vandiver 3e57d66632 sentry: Drop unused integrations argument. 2021-07-15 15:01:43 -07:00
PIG208 c03b9c95ad request: Store client information using ZulipRequestNotes.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
2021-07-14 12:01:07 -07:00
PIG208 8eb2c3ffdb request: Move realm from the request to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 3f9a5e1e17 request: Refactor to record rate limit data using ZulipRequestNotes.
We will no longer use the HttpRequest to store the rate limit data.
Using ZulipRequestNotes, we can access rate_limit and ratelimits_applied
with type hints support. We also save the process of initializing
ratelimits_applied by giving it a default value.
2021-07-14 12:01:07 -07:00
Tim Abbott 0bf13ce5db portico: Add Technical University of Munich case study.
This isn't linked yet, since the main place we'll link to is isn't
added yet.
2021-07-09 13:04:38 -07:00
Alex Vandiver a985284b5d sentry: Set environment to `staging` on staging hosts.
Splitting exceptions out between staging and production provides
useful filtering.
2021-07-09 10:54:19 -07:00
Mateusz Mandera 85cbdc8904 rate_limit: Add rate limiting of ZulipRemoteServer. 2021-07-08 15:55:02 -07:00
Mateusz Mandera b9056d193d rate_limit: Implement IP-based rate limiting.
If the user is logged in, we'll stick to rate limiting by the
UserProfile. In case of requests without authentication, we'll apply the
same limits but to the IP address.
2021-07-08 15:46:52 -07:00
Mateusz Mandera 3b4f8cc85b saml: Add setting to skip the "continue to registration" page.
It's a smoother Just-In-Time provisioning process to allow
creating the account and getting signed in on the first login by the
user.
2021-07-08 15:21:20 -07:00
Anders Kaseorg 98194b9316 blueslip: Remove SAVE_FRONTEND_STACKTRACES setting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Anders Kaseorg 07fef56c74 logging_handlers: Remove STAGING_ERROR_NOTIFICATIONS setting.
Running notify_server_error directly from the logging handler can lead
to database queries running in a random context.  Among the many
potential problems that could cause, one actual problem is a
SynchronousOnlyOperation exception when running in an asyncio event
loop.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Mateusz Mandera cee4da64fa saml: Don't raise AssertionError if no name is provided in SAMLResponse.
This is an acceptable edge case for SAML and shouldn't raise any errors.
2021-07-07 09:23:21 -07:00
Anders Kaseorg ee8724e436 docs: Correct Big Blue Button to BigBlueButton.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:00 -07:00
Anders Kaseorg 37f98fdec8 zproject: Add prod_settings mypy stub, aliasing prod_settings_template.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 09:53:41 -07:00
Damian Parrino caea7f167d help: Reorganize articles on editing topics.
* Move content on moving topics between streams to a dedicated
  article. We advertise it as "move content" to hint that one can move
  messages or split topics, and link to it.

* This deletes change-the-topic-of-a-message, because the same content
  is already covered in rename-a-topic.

* This commit mostly just moves content between articles. Most of that
  content was redundant with the first few paragraphs of the surviving
  "rename a topic" article. The former "This is useful for" se ntence
  was adapted to the remaining article.

* This commit also adds a redirect for the removed article, and
  updates related links.
2021-06-29 13:52:06 -07:00
Tim Abbott 06121e1f69 urls: Add redirect for community_topic_edits rename. 2021-06-16 15:07:16 -07:00
Mateusz Mandera b1da5d9d76 ldap: Prevent creation of users with malformed email value. 2021-06-11 17:30:46 -07:00
Suyash Vardhan Mathur 001435a297 refactor: Rename and move app_filters.py.
Moved the code in zerver/templatetags/app_filters.py
to more intuitive location and name at
zerver/lib/templates.py.
2021-06-11 07:43:22 -07:00
Mateusz Mandera c54b48452d saml: Support syncing custom profile fields with SAML attributes.
Fixes #17277.

The main limitation of this implementation is that the sync happens if
the user authing already exists. This means that a new user going
through the sign up flow will not have their custom fields synced upon
finishing it. The fields will get synced on their consecutive log in via
SAML in the future. This can be addressed in the future by moving the
syncing code further down the codepaths to login_or_register_remote_user
and plumbing the data through to the user creation process.

We detail that limitation in the documentation.
2021-06-08 14:47:33 -07:00
Mateusz Mandera 00c7ac15df saml: Define a TypedDict for the type of IdP settings dict.
The old type in default_settings wasn't right - limit_to_subdomains is a
List[str]. We define a TypeDict for capturing the typing of the settings
dict more correctly and to allow future addition of configurable
attributes of other non-str types.
2021-06-08 14:30:21 -07:00
Anders Kaseorg 61e1e38a00 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-07 17:57:51 -07:00
Gaurav Pandey 9b696cf212 api: Expose event_queue_longpoll_timeout_seconds in /register.
Rename poll_timeout to event_queue_longpoll_timeout_seconds
and change its value from 90000 ms to 90 sec. Expose its
value in register api response when realm data is fetched.
Bump API_FEATURE_LEVEL to 74.
2021-06-05 07:37:19 -07:00
akshatdalton 7df62ebbaf settings: Make `MAX_MESSAGE_LENGTH` a server-level setting.
This will offer users who are self-hosting to adjust
this value. Moreover, this will help to reduce the
overall time taken to test `test_markdown.py` (since
this can be now overridden with `override_settings`
Django decorator).

This is done as a prep commit for #18641.
2021-06-03 09:26:28 -07:00
Gaurav Pandey f67fc37f7d settings: Rename MAX_EMOJI_FILE_SIZE.
Rename MAX_EMOJI_FILE_SIZE to
MAX_EMOJI_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 73059dd0c7 settings: Rename MAX_LOGO_FILE_SIZE.
Rename MAX_LOGO_FILE_SIZE to
MAX_LOGO_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 2ddc62e9b3 settings: Rename MAX_ICON_FILE_SIZE.
Rename MAX_ICON_FILE_SIZE to
MAX_ICON_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 0f6bb210a6 settings: Rename MAX_AVATAR_FILE_SIZE.
Rename MAX_AVATAR_FILE_SIZE to
MAX_AVATAR_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Mateusz Mandera b202116594 auth: Extract sync_user_profile_custom_fields function from ldap sync.
This logic will be reusable for analogical sync in other auth codepaths.
2021-06-01 08:32:50 -07:00
Mateusz Mandera 47a4a9193f auth: Make full_name_validated attr of generic OIDC configurable.
This will be useful for deployments that want to just use the full name
provided by the IdP and thus skip the registration form. Also in
combination with disabling name changes in the organization, can force
users to just use that name without being able to change it.
2021-05-28 09:50:23 -07:00
Mateusz Mandera e17758f8ad auth: Implement a generic OpenID Connect backend.
Fixes #11939.
2021-05-28 09:43:06 -07:00
Alex Vandiver 54c222d3f8 settings: Support arbitrary database user and dbname.
This adds basic support for `postgresql.database_user` and
`postgresql.database_name` settings in `zulip.conf`; the defaults if
unspecified are left as `zulip`.

Co-authored-by: Adam Birds <adam.birds@adbwebdesigns.co.uk>
2021-05-25 13:46:58 -07:00
Mateusz Mandera 1df725e6f1 settings: Silence CryptographyDeprecationWarning spam from a dependency. 2021-05-23 13:31:55 -07:00
Mateusz Mandera cd88a6e6fb tests: Rename APPLE_JWK to a generic name for re-use in other tests.
This can be re-used for the JWK in OIDC implementation in the follow-up
commits.
2021-05-23 13:30:17 -07:00
Alex Vandiver 670c7e7ba4 settings: Remove now-unnecessary EMAIL_DELIVERER_DISABLED setting. 2021-05-18 12:39:28 -07:00
Alex Vandiver 82797dd53c settings: Standardize the name of the deliver_scheduled_messages logs.
This makes it match its command name, and other logfile name.
2021-05-18 12:39:28 -07:00
Anders Kaseorg 79c087471c jinja2: Serialize page-params with orjson.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-18 12:36:15 -07:00
Anders Kaseorg e015f3ed7d docs: Correct “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:15:38 -07:00
Vishnu KS 051dab58ea navbar: Add gear menu advertisement for sponsoring zulip. 2021-05-12 10:21:48 -07:00
Alex Vandiver 0f1611286d management: Rename the deliver_email command to deliver_scheduled_email.
This makes it parallel with deliver_scheduled_messages, and clarifies
that it is not used for simply sending outgoing emails (e.g. the
`email_senders` queue).

This also renames the supervisor job to match.
2021-05-11 13:07:29 -07:00
Anders Kaseorg 544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
Anders Kaseorg d0c6f4f400 python: Strip leading and trailing spaces from docstrings.
This is enforced by Black ≥ 21.4b0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 22:42:39 -07:00
Mateusz Mandera 4006bb5153 auth: Improve display of errors when user needs to reset password.
Raising jsonableError in the authentication form was non-ideal because
it took the user to an ugly page with the returned json.

We also add logging of this rare occurence of the scenario being
handled here.
2021-05-07 09:10:45 -07:00