Previously, we included a special subscribe button in new stream
notifications, but that had 2 problems:
(1) The subscribe button would render badly if the stream was renamed.
(2) There wasn't an easy way to look at the stream when deciding
whether to subscribe.
This fixes the second problem, but not really the first.
This adds support for only allowing normal users with account age
equal or greater than a "waiting period" threshold to create streams;
this is useful for open organizations that want new members to
understand the community before creating streams.
If create_stream_by_admins_only setting is set to True, only admin users
were able to create streams. Now normal users with account age greater
or equal than waiting period threshold can also create streams.
Account age is defined as number of days passed since the user had
created his account.
Fixes: #2308.
Tweaked by tabbott to clean up the actual can_create_streams logic and
the tests.
First step in cleaning up populate_db.create_streams and
bulk_create.bulk_create_streams. Part of a series of commits to remove
Realm.domain from populate_db.
There is a change in Django 1.10 due to which whenever the password
of the user is changed the session hash changes. This change affects
us because we cache user profile objects and these cached objects need
to be refreshed. However, the signal sent by Django in which objects are
refreshed fails to refresh the cache for Tornado because it uses a
different cache prefix.
Note: Backend tests are not affected because they don't rely on Tornado.
Expose `is_mentioned` in `message` dict which contains
boolean value about our account is mentioned in the message
content or not.
This is already available via `flags`, but it seems worth making this
data point more explicit, given its importance in writing bots.
Fixes#2667.
Adding more additional information about user profile to
`zerver.views.pointer.get_profile_backend`, like `user_id`,
`full_name`, `email`, `is_bot`, `is_admin`, and `short_name` of the
user.
Zulip doesn't previously make use of the standard Django is_staff flag
(in that the Django admin site is disabled), but since conceptually
the /activity page would be part of the Django admin site if we were
using it (i.e. for server-level administrators), it makes sense to key
off of that rather than the previous, fragile, check for the realm
domain name.
Adding a reaction is now a PUT request to
/messages/<message_id>/emoji_reactions/<emoji_name>
Similarly, removing a reaction is now a DELETE request to
/messages/<message_id>/emoji_reactions/<emoji_name>
This commit changes the url and updates the views and tests.
This commit also adds a test for invalid emoji when removing reaction.
This includes making the default stream description setting into a
dict. That is an API change; we'll discuss it in the changelog but it
seems small enough to be OK.
With some small tweaks by tabbott to remove unnecessary backwards
compatibility code for the settings.
Fixes#2427.
This change adds support for displaying inline open graph previews for
links posted into Zulip.
It is designed to interact correctly with message editing.
This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.
By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.
Eventually, we may want to make this manageable via a (set of?)
per-realm settings. E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
This automates including of markdown files under
`templates/zerver/help/` to be tested by `test_public_urls` (in
zerver/tests/test_signup.py).
Fixes#2465.
This can be useful in scenarios where the network doesn't support
websockets. We don't include it in prod_settings_template.py since
it's a very rare setting to need.
Fixes#1528.
A few functions had arguments removed without having their type
annotations updated accordingly. As a result mypy version 0.4.6
thinks that the first type in the annotation is supposed to be
the type of `self`, a new mypy feature which we are not intending
to use here.
This commit adds support for removing reactions via DELETE requests to
the /reactions endpoint with parameters emoji_name and message_id.
The reaction is deleted from the database and a reaction event is sent
out with 'op' set to 'remove'.
Tests are added to check:
1. Removing a reaction that does not exist fails
2. When removing a reaction, the event payload and users are correct
The markdown files under templates/zerver/help/ are technically not
templates in the standard sense, and thus should not be being
checked with this code path.
(We probably do want to add a test to make sure they all render fine,
but that can be its own project.)
Modify backend test of create_streams_if_needed so that the newly
created streams have descriptions.
Modify casperjs test of filling out stream_creation_form so that
the newly created stream has a description.
Fixes: #2428.
- Add base tornado test case class.
- Add test for websocket connection.
- Add test for websocket authentication.
- Add test for sending private message with websocket.
- Add test for sending stream message with websocket.
Fixes#2230
This commit adds the following:
1. A reaction model that consists of a user, a message and an emoji that
are unique together (a user cannot react to a particular message more
than once with the same emoji)
2. A reaction event that looks like:
{
'type': 'reaction',
'op': 'add',
'message_id': 3,
'emoji_name': 'doge',
'user': {
'user_id': 1,
'email': 'hamlet@zulip.com',
'full_name': 'King Hamlet'
}
}
3. A new API endpoint, /reactions, that accepts POST requests to add a
reaction to a message
4. A migration to add the new model to the database
5. Tests that check that
(a) Invalid requests cannot be made
(b) The reaction event body contains all the info
(c) The reaction event is sent to the appropriate users
(d) Reacting more than once fails
It is still missing important features like removing emoji and
fetching them alongside messages.
Refactor list_to_streams and create_streams_if_needed to take a list
of dictionaries, instead of a list of stream names. This is
preparation for being able to pass additional arguments into the
stream creation process.
An important note: This removes a set of validation code from the
start of add_subscriptions_backend; doing so is correct because
list_to_streams has that same validation code already.
[with some tweaks by tabbott for clarity]
Previously, the key prefix was based on the process id due to which
the JS tests couldn't properly flush user profiles from the cache as
our application spans over multiple processes. This problem becomes
apparent when in json_change_settings view after changing the user_profile
the tornado views continue to get the cached user profile corresponding
to their process id.
Clean up the instances of self.assertIn("string", result.content.decode("utf-8")),
and replace them with self.assert_in_response("string").
Fixes: #2313
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.
If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.
With some tweaks to the output from tabbott.
Fixes#1441.
Previously, we rejected the HEAD requests that the trello integration
uses to check if the server accepts the integration.
Add decorator for returning 200 status code if request is HEAD.
Fixes: #2311.
Django reverts all the changes after running a test but the
client cache retained the deleted value, this caused the
subsequent tests to fail due to invalid foreign key constraints.
This commit fixes the issue by prefixing the cache name in
client cache with KEY_PREFIX which is bounced after every test.
Updates the HTML docs to match changes to the Desk.com website,
including all new screenshots for the custom action workflow.
Tests four types of messages that could be sent as notifications from
Desk.com. Desk.com allows an account administrator to send any text
in a custom action, so there isn't a standard format.
Custom actions send URL-encoded POST data, the test fixtures contain
URL-encoded text like what could be sent by a custom action configured
as described in the Zulip Integrations documentation to post a new
message to a stream. (See also #2169, errors in this documentation.)
New zerver.tests.webhooks.test_deskdotcom.DeskDotComHookTests:
* Static text: minimal plain text string
* Case updated: activity alert with link to a Desk.com case and message
* Unicode text Italian: activity alert with message in Italian
* Unicode text Japanese: activity alert with message in Japanese
Each posts a new message in the deskdotcom stream.
Tested on Ubuntu 14.04. I created the fixtures with Emacs, I would
appreciate if someone can check that the Italian and Japanese messages
look ok. I used the same text for a live test and it displayed correctly.
Fixes#2031
Add is_truncated param for cases when we know that there were more commits
but we dont know actual number and want just inform about existing of the rest.
bulk_create.bulk_create_users is only called by initialize_voyager_db.py and
populate_db.py, both of which only have realms with single domains.
Part of a larger project to remove the Realm.domain field.
No change in behavior. The for_you block had already been removed in
portico.html long ago in a6889080ce. The
contents of the block are still present in the non-portico 404.html
and 5xx.html error pages.
Currently refactoring the Realm and RealmAlias portions of models.py to no
longer use Realm.domain. Move various functions related and not-related to
the two model objects to more conventional places in the file.
No change to behavior.
Previously, if a new message arrived between when a user is subscribed
to the default streams and when the user's initial messages are
queried, we would try to create two UserMessage rows for the same
Message, resulting in an IntegrityError crash. We fix this and add a
test for that race condition.
This migration ensures that all historically uploaded files from
before we started tracking files in the Zulip database via the
Attachment model have Attachment objects.
This has been tested by tabbott against a production server to ensure
that it results in all old uploaded files having corresponding
attachment objects.
Merging this change is a key prerequisite for making our adding
attachment access controls in an enforcing fashion.
In Django 1.10, the get_token function returns a salted version of
csrf token which changes whenever get_token is called. This gives
us wrong result when we compare the state after returning from
Google authentication servers. The solution is to unsalt the token
and use that token to find the HMAC so that we get the same value
as long as t he token is same.
Needed in case the user was allowed to join the realm when they got the
confirmation email, but is no longer allowed to do so. Check was previously
applied to invited users (those with a prereg_user.referred_by), and is now
applied regardless of how they get to accounts_register.
Does change/fix behavior in various corner cases when the domain passed in
to HomepageForm and subdomain passed in to HomepageForm correspond to
different realms.
If the user comes in to HomepageForm with a set subdomain, use that to
determine the signup realm instead of the email address.
In the non-REALMS_HAVE_SUBDOMAINS case, still allow using the email address
if no subdomain is passed.
Ensure domain and subdomain correspond to the same realm when being passed
to forms.HomepageForm. Previously this was not the case when e.g. we got
here via the /register/<domain> endpoint.
This also effectively disables the register/<domain> endpoint when
REALMS_HAVE_SUBDOMAINS, or rather, foo.server.org/register/bar.com will try
to register you for the realm with string_id foo rather than realm with
domain bar.com.
`django.contrib.auth.get_user` function is updated in Django 1.10, due to
which everytime we update the password of the user the password hash changes.
This causes authentication failure. Previously, our code worked correctly
because we use our own session middleware and the `get_user` code had a
conditional statement which allowed our code to bypass the authentication
code.
Previously, the way that render_messages was calling bugdown meant
that the preview feature didn't have access to realm data like the
list of users or streams, resulting in previews for those elements
being wrong.
Now render_message_backend uses zerver.lib.render_markdown to render
messages correctly.
[Commit message tweaked and test added by tabbott]
No change to behavior. non_mit_mailing_list never returned False, so it was
never possible to reach the line "Otherwise, the user is an MIT mailing
list, and .."
send_event() expects a list of user ids (ints) except for the special case
of messages. This commit:
1. Fixes this in the call to send_event() in do_send_typing_notification()
2. Renames the variables in do_send_typing_notification() to better reflect
their content (for example, recipient_ids instead of recipients).
3. Renames the id field in the dicts sent in the typing event body (sender,
recipients) to user_id.
4. Adds assertions to the tests to verify that the tornado event user ids
are the same as the recipients in the event body.
5. Adds assertions to the tests to verify that the tornado event user
ids and the recipient user ids (in the event body) are the same as the
expected user ids (obtained from the emails using
get_user_profile_by_email)
6. Changes all assertTrues to assertEquals in the tests
This fixes#2151.
This makes it possible to configure only certain authentication
methods to be enabled on a per-realm basis.
Note that the authentication_methods_dict function (which checks what
backends are supported on the realm) requires an in function import
due to a circular dependency.
In fe1ba6f3eb, we change our auth
decorators etc. to use request.POST/request.GET rather than the (now
removed request.REQUEST). This broke sending messages via our
websockets codepath, because we failed to update the artificial
requests we generate to use request._post as well.
We recently made it so that a cross-realm bot can only send
messages to one realm at a time. (It can send to a realm
outside of its offical realm, but only one of them.) This
test adds coverage for that.
Disallow Realm.string_id's like "streams", "about", and several hundred
others. Also restrict string_id's to be at least 3 characters long, and only
use characters in [a-z0-9-].
Does not restrict realms created by the create_realm.py management command.
Previously realm_name and realm_subdomain defalted to None, which when
posted to /accounts/register, are submitted as u'None'. "None" is an invalid
Realm.subdomain, since subdomains can't have capital letters.
Before it was in UserSignUpTest, now it is in RealmCreationTest. The diff
makes it look like test_user_default_language is the target of the move,
but it isn't.
If a stream is public, we now send notifications to all realm users
if the name or description of the stream changes. For private
streams, the behavior remains the same.
We do this by introducing a method called
can_access_stream_user_ids().
(showell helped with this fix)
Fixes#2195
We use the queries_captured() context manager in our tests
to capture queries that happen during certain actions. Usually
we use the list of queries to validate that we're not doing
too many database calls.
For most tests the database calls are fairly deterministic,
but SAVEPOINT-related things seem to be more random, and the
number of savepoints is usually not relevant to what the test
is trying to prevent, which is more serious problems like
O(N) database fetches.
This (1) removes the check on whether the domain of the email matches
the Realm.domain of an existing realm and (2) avoids setting `realm =
get_realm(domain)` in the realm creation flow, which would cause the
wrong code path to be followed in the event that the domain in a
user's email address happens to match a deactivated realm.
Removes the uniqueness constraint on RealmAlias.domain, and adds a function
can_add_alias that checks for uniqueness conditional on
settings.REALMS_HAVE_SUBDOMAINS.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.
We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.
This commit adds a database migration.
This test seems intended to verify registration in the case of a
unique completely open domain; but because of the mit.edu realm, it
instead tested that a logic bug in the non-subdomains case was
present.
We now send dictionaries for cross-realm bots. This led to the
following changes:
* Create get_cross_realm_dicts() in actions.py.
* Rename the page_params field to cross_realm_bots.
* Fix some back end tests.
* Add cross_realm_dict to people.js.
* Call people.add for cross-realm bots (if they are not already part of the realm).
* Remove hack to add in feedback@zulip.com on the client side.
* Add people.is_cross_realm_email() and use it in compose.js.
* Remove util.string_in_list_case_insensitive().
In preparation for a change to do_create_realm where we will use the
database default for restricted_to_domain rather than computing it within
do_create_realm, and due to which do_create_realm will no longer know
whether we are creating an open realm or not.
Step 0 of a two step process:
1. Replace all occurances of get_realm(domain) with
get_realm_by_string_id(string_id)
2. Rename get_realm_by_string_id to get_realm.
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
Does a database migration to rename Realm.subdomain to
Realm.string_id, and makes Realm.subdomain a property. Eventually,
Realm.string_id will replace Realm.domain as the handle by which we
retrieve Realm objects.
We now simply exclude all cross-realm bots from the set of emails
under consideration, and then if the remaining emails are all in
the same realm, we're good.
This fix changes two behaviors:
* You can no longer send a PM to an ordinary user in another realm
by piggy-backing a cross-realm bot on to the message. (This was
basically a bug, but it would never manifest under current
configurations.)
* You will be able to send PMs to multiple cross-realm bots at once.
(This was an arbitrary restriction. We don't really care about this
scenario much yet, and it fell out of the new implementation.)
We can currently send a PM to a user in another realm, as long
as we copy a cross-realm bot from the same realm. This loophole
doesn't yet affect us in practice--all cross-realm bots are
generally configured for the "admin" realm like the old zulip.com--
but we should lock it down in a subsequent commit.
Having each condition in a separate test was confusing to read,
especially since the tests were doing inconsistent setup, sometimes
calling user2 the user from 2.example.com realm and other times
calling user2 the cross-bot realm, etc.
Note that we still need the equivalent function in our
user-facing API, so there is not much code removal yet.
(Also, we will probably always keep this in our API,
as bot authors will usually just want a simple endpoint
here, whereas our client code gets page_params and events.)
Previously, we used to create one Google OAuth callback url entry
per subdomain. This commit allows us to authenticate subdomain users
against a single Google OAuth callback url entry.
- To avoid redefining migrate manage command is added new application
configuration class which emit post_migration signal. This signal
require models module inside application and defined AppConfig
Instance as signal sender. Documentation here:
https://docs.djangoproject.com/en/1.8/ref/signals/#post-migrate.
- Add AppConf subclass to __init__ zerver app file to make apllication
load it by default.
Fixes#1084.
For some reason, we use 'load' function but it doesn't exist in the JWT
library code. This commit updates the code to use the correct interface
of the JWT library.
The signature verification is done by the decode function.
This is a first step towards implementing a message retention policy
feature.
- Add Realm model message_retention_days field to setup
messages expired period for realm.
- Add migration.
- Add tool to get expired messages for each Realm.
- Add tests to cover tool for getting expired messages.
Now that we have updated python-markdown, we remove the deprecated
safe_mode. We used safe_mode to escape raw html, so now instead we
pass in an EscapeHtml markdown extension to the markdown engine.
See https://pythonhosted.org/Markdown/release-2.6.html for details on
the deprecation.
Fixes: #2037 (also addresses the remaining piece of #2043).
Adds additional tests in bugdown-data.json to check that we escape raw
html properly with script tags.
Many of these tests are based on the bleach "advanced" test suite.
Passes the allowed domains for a realm to the frontend, via
page_params.domains. Groundwork for allowing users to add and
remove domains via the admin setting page, rather than via the
realm_alias.py management command.
This is a preliminary step towards eliminating the realm.domain field
in favor of realm.subdomain. Includes a database migration to create
these for existing realms.
This adds a medium (500px) size avatar thumbnail, that can be
referenced as `{name}-medium.png`. It is intended to be used on the
user's own settings page, though we may come up with other use cases
for high-resolution avatars in the future.
This will automatically generate and upload the medium avatar images
when a new avatar original is uploaded, and contains a migration
(contributed by Kirill Kanakhin) to ensure all pre-existing avatar
images have a medium avatar.
Note that this implementation does not provide an endpoint for
fetching the medium-size avatar for another user.
[substantially modified by tabbott]
When we added data on never_subscribed streams to what
populate_subscribers is called on, we failed to add the corresponding
data on subscribers to email_dict, the mapping of user IDs to emails
for the subscribers.
Because in the Zephyr world, stream names can be a secret, and also
Zephyr mirroring tends to involve many thousands of streams, we
shouldn't send this data.
This is some of the code we'd need if we wanted to have Zulip generate
avatars for things. Since it is so little useful code, and it's not
clear we will need this feature ever, we can remove this code to make
the codebase less confusing. It'd be easy to dig this out of history
if we ever want it.
Fixes#2101.
- Add tests for SEND_MISSED_MESSAGE_EMAILS_AS_USER is False (the
default!).
- Reorganized test case code by removing repeated parts of code,
improving code style and moving common parts to separate class
methods.
Fixes#1697.
POST to /typing creates a typing event
Required parameters are 'op' ('start' or 'stop') and 'to' (recipient
emails). If there are multiple recipients, the 'to' parameter
should be a JSON string of the list of recipient emails.
The event created looks like:
{
'type': 'typing',
'op': 'start',
'sender': 'hamlet@zulip.com',
'recipients': [{
'id': 1,
'email': 'othello@zulip.com'
}]
}
We now send peer_remove events to folks who have never subscribed
to the streams (except for private streams and zephyr).
We also use logic that is more similar to how
bulk_add_subscriptions() works.
There are two reasons for this change. First, we want to be
consistent with notify_subscriptions_added(), which doesn't
handle "peer" events. Second, we want to fix this code in a
subsequent commit not to do one user at a time, which is
inefficient.
Compare the hash of 'zilencer/management/commands/populate_db' with
'var/populate_db_hash' and 'tools/setup/postgres-init-test-db' with
'var/postgres_init_test_db_hash' and if any comparison fails rebuild
the database.
With fixes from tabbott.
As best I can tell, this option was completely confused in two ways:
* The name is confusing; it actually controls whether we _clear_ the
timeout associated with the current handler
* It's not clear why one would not want it to be unconditionally true.
From reading the history, I'm pretty sure I had just misread the code
when I created this.
And this caused a real bug; a later refactoring caused us to basically
never cancel the timeouts, which in turn resulted in 90% of all events
traffic being hearbeats with a much lower frequency (~5s) than the
intended 45s. Removing this code fixes that nasty bug.
merge_vars is the user meta data we store in mailchimp. This commit changes
what we store from realm.domain to realm.id, since realm.domain is being
deprecated, and changes the OPTIN_TIME to a nicer format.
This distinguishes between YouTube Videos and Image Previews by adding
a particular “youtube-video” class to the preview along with changing
the title to the video ID rather than the link. This serves to allow
the lightbox to ID when a lightbox preview should be treated like a
YouTube video rather than an image preview.
This also modifies the tests in bug down to expect a youtube-video class
along with the title to just be the video ID on YouTube rather than the
entire URL link.
This changes `from django.utils.importlib import import_module` to
`from importlib import import_module`, as `django.utils.importlib` will
be removed in django version 1.9.
The changes that required us to fork this extension had been merged
into upstream CodeHilite, so we can remove it and switch to using the
version that comes with python-markdown.
This updates Bugdown to reflect the changes in the updated
markdown. In particular, we now pass a default config object in the
__init__ for the Bugdown extension, update the make_md_engine function
to take kwargs as opposed to a config list, and have UListProcessor
inherit from ulist as opposed to olist (which no longer works).
We update the (forked from upstream) fenced_code extension's
makeExtension to take args and kwargs, and update
FencedBlockPreprocessor __init__ method with updated Codehilite
arguments.
We update the (forked from upstream) Codehilite extension to
mirror the logic with the latest upstream Codehilite:
Add parse_hl_lines function
update makeExtension to take args and kwarfs instead of config
list
Add regex for highlight lines
use linenums instead of linenos
use get_formatter_by_name instead of HtmlFormatter
user get_lexer_by_name instead of TextLexer
add hl_lines and use_pygments arguments to the codehlite
constructor
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / PR #id title'.
Modify some test fixtures for better coverage.
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / PR #id title'.
Modify some test fixtures for better coverage.
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / MR #id title'.
Modify some test fixtures for better coverage.
Fixes: #1883.
Rename:
PUSH_COMMITS_LIMIT to COMMITS_LIMIT
PUSH_COMMIT_ROW_TEMPLATE to COMMIT_ROW_TEMPLATE
PUSH_COMMITS_MORE_THAN_LIMIT_TEMPLATE to COMMITS_MORE_THAN_LIMIT_TEMPLATE
In order to use the latest version of psycopg2 with the latest version
of sqlalchemy we must integrate the changes made in the 2.4.6 release of
psycopg2. See
c86ca7687f
and the release notes for psycopg2 2.4.6.
Change the CountStat object to take an is_gauge variable instead of a
smallest_interval variable. Previously, (smallest_interval, frequency)
could be any of (hour, hour), (hour, day), (hour, gauge), (day, hour),
(day, day), or (day, gauge).
The current change is equivalent to excluding (hour, day) and (day, hour)
from the list above.
This change, along with other recent changes, allows us to simplify how we
handle time intervals. This commit also removes the TimeInterval object.
With reactions and other upcoming features, we'll be adding several
places where we need to check whether a particular user can access a
particular message. It's best to just have a single helper function
for this purpose that we can use everywhere.
Previously, we sent users to an "invite your friends" page after they
created an organization. This commit removes that step in the flow and sends
users directly to the home page. We also remove the now-unused
initial_invite_page.html template, initial_invite.js (which pre-filled the
invite emails with characters from literature), and the /invite URL route.
Change the parameter name of some functions from 'md' to 'content',
since the name 'md' seems to be the reason why this parameter was
wrongly annotated.
Previously, we suggested running
`python -c import zerver.tests.test_mytest`
when importing test_mytest failed, which doesn't work.
This commit adds the missing quotes, making it
`python -c 'import zerver.tests.test_mytest'`
test_settings.py was setting EXTERNAL_HOST after importing settings.py,
which has several variables (like SERVER_URI) that are computed from
EXTERNAL_HOST.
[tweaked by tabbott to add comments explaining the story here].
Adds a new field org_type to Realm. Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level. Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.
Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.
Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.
The database defaults have not been changed, which allows our testing code
to work unchanged.
[includes some HTML/CSS work by Brock Whittaker to make it look nice]
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state. Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.
We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.
Fixes#1221.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
aggregating user/stream level stats into realm level stats
* A management command for pulling the data
Note that counts.py was added to the linter exclude list due to errors
around %%s.
The command to render old messages now looks for all messages
not matching the bugdown version, and it no longer directly calls
into model code. We should still be extremely cautious about
using this code.
This pulls message-related code from models.py into a new
module called message.py, and it starts to break some bugdown
dependencies. All the methods here are basically related to
serializing Message objects as dictionaries for caches and
events.
extract_message_dict
stringify_message_dict
message_to_dict
message_to_dict_json
MessageDict.to_dict_uncached
MessageDict.to_dict_uncached_helper
MessageDict.build_dict_from_raw_db_row
MessageDict.build_message_dict
This fix also removes a circular dependency related
to get_avatar_url.
Also, there was kind of a latent bug in Message.need_to_render_content
where it was depending on other calls to Message to import bugdown
and set it globally in the namespace. We really need to just
eliminate the function, since it's so small and used by code that
may be doing very sketchy things, but for now I just fix it. (The
bug would possibly be exposed by moving build_message_dict out to the
library.)
I move these three functions to lib/cache.py:
to_dict_cache_key_id
to_dict_cache_key
flush_message
This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
This removes some unused code on Attachment. Some of these
methods might be useful in a manage.py shell, but without tests,
they are not very trustworthy, and the Attachment model isn't that
complicated to write raw Django queries against.
This moves the logic for renaming a stream to the REST API
update_stream_backend method, eliminating the legacy API endpoint for
doing so.
It also adds a nice test suite covering international stream names.
This improves Google and JWT auth as well as the registration
codepath to log something if the wrong subdomain is encountered.
Ideally, we'd have tests for these, and code to make the Google and JWT
auth cases show a clear error message.
This ensures that everything is using the correct subdomain for
requests. While it probably wouldn't be a real security problem for
the wrong subdomain to work, this enforcement is essential to catching
bugs in the product and users' API scripts.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.
This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
(but does not restrict the API; that will be tightened in a future commit).
Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).
[substantially modified by tabbott as part of merging]
This sets the “title” attribute on the image to the actual title of
the image specified by the user in their markdown, rather than just
the URL of the full link to it.
This was the original way to send messages via the Zulip API in the
very early days of Zulip, but was replaced by the REST API back in
2013.
Fixes: #730.
Sorry, couldn't resist wording the commit message like that. :)
The remove_unreachable() method on Message was no longer being
used, and the commit history made it fairly clear we won't need it
in the future.
We can now rely on UserProfile.last_reminder being time zone
aware, or even if it isn't, it's a self-correcting problem the
first time a reminder is sent. (It's a non-problem to be off
by a few timezones if somebody still has an old value there, because
they will still be outside the 1-minute nag window even with the
timezone disparity.)
This is partly a concession to testing; it's really hard to test
that we are flushing the cache properly if tests need to look
at a global variable in models.py that can be re-assigned on every
request. Extracting this function makes it easy for tests to know
whether a domain is in the local cache.
We currently do
var invite_suffix = "{{invite_suffix}}";
in javascript in the initial_invite_page.html template.
This sets invite_suffix to "{{invite_suffix}}" when the template is rendered
without invite_suffix in the params, rather than to "" as intended. This
later causes problems in the invite_email validator in initial_invite.js.
We no longer use all the alert words for all the users in the
entire realm when we look for alert words in a newly sent/edited
message. Now we limit the search to only all the alert words
for all the users who will get UserMessage records. This will
hopefully make a big difference for big realms where most messages
are only sent to a small subset of users.
The bugdown parser no longer has a concept of which users need which
alert words, since it can't really do anything actionable with that info
from a rendering standpoint.
Instead, our calling code passes in a set of search words to the parser.
The parser returns the list of words it finds in the message.
Then the model method builds up the list of user ids that should be
flagged as having alert words in the message.
This refactoring is a little more involved than I'd like, but there are
still some circular dependency issues with rendering code, so I need to
pass in the rather complicated realm_alert_words data structure all the way
from the action through the model to the renderer.
This change shouldn't change the overall behavior of the system, except
that it does remove some duplicate regex checks that were occurring when
multiple users may have had the same alert word.
We now use render_incoming_message() to render all incoming
new messages (sends/edits), so that they will get the same treatment.
This change also establishes do_send_messages() as the code
path to get new messages rendered. It removes some
logic from check_message() that only happened on certain code paths
for sending messages, and which would only detect failures by
expensively rendering messages, so it wasn't much of a guard.
This change also helps to phase out maybe_render_content(), which
deepens the call stack without providing much clarity to the reader,
since it's behavior is so variable.
Finally, this sets up to fix a flaw in the way we compute which
users have alert words in their messages (in a subsequent commit).
If you supplied an unrecognizable address to our email system,
or you had EMAIL_GATEWAY_PATTERN configured wrong,
the get_missed_message_token_from_address() used to crash
hard and cryptically with a traceback saying that you can't
call startswith() on a None object.
Now we throw a ZulipEmailForwardError exception. This will
still lead to a traceback, but it should be easier to diagnose
the problem.
In our email mirror, we have a special format for missed
message emails that uses a 32-bit randomly generated token
that we put into redis that is then prefixed with "mm" for
a total of 34 characters.
We had a bug where we would mis-classify emails like
mmcfoo@example.com as being these system-generated emails
that were part of the redis setup.
It's actually a little unclear how the bug in the library
function would have manifested from the user's point of view,
but it was definitely buggy code, and it's possibly related in
a subtle way to an error report we got from a customer where
only one of their users, who happened to have a name like
mmcfoo, was having problems with the mirror.
paxapy
Tim Abbott
Tomasz Kolek
vaibhav
Rishi Gupta
Umair Khan
Sampriti Panda
Kracekumar R
Igor Tokarev
Harshit Bansal
Rafid Aslam
reyha
Bojidar Marinov
reallyly
Jason Le
Zac Pullar-Strecker
Arpith Siromoney
Tomonori Murakami
Joy Chen
anirudhjain75
root
Bickio
K.Kanakhin
Mikebarson
Vamshi Balanaga
sidhant bhavnani
bulat22101
Noel Tautges
AZtheAsian
Reid Barton
nikolay
Sidhant Bhavnani
Alex Huang
Tommy Ip
sylvan1
Calvin Lee
Steve Howell
Anders Kaseorg
Vladislav Manchev
hackerkid
Pweaver (Paul Weaver)
Tim Abbott
Tomasz Kolek
Feorlen
sinwar
Mohsen Ibrahim
Meena Rajan
umkay
Kevin Chen
Rishi Gupta
Brock Whittaker
Amy Liu
Laura Hampton
Eklavya Sharma
Kartik Maji
sonali0901
baali
Sahil Dua
Supriya
sunnypalace