The goal of this library is to make it a lot easier to prevent bugs
like CVE-2017-0881 by having all of our views logic for fetching a
stream go through a couple carefully tested code paths.
A bug in Zulip's implementation of the "stream exists" endpoint meant
that any user of a Zulip server could subscribe to an invite-only
stream without needing to be invited by using the "autosubscribe"
argument.
Thanks to Rafid Aslam for discovering this issue.
Apparently, we weren't returning the `json_error`, resulting in users
encountering this condition receiving a 500, rather than the proper
40x error.
This fixes a regresion introduced in 9ae68ade8b.
Previously, if you searched for ':offi..' you would see both 🏢 and
:office_building: as possible completions, both of which are shortcodes for
the same unicode codepoint (and hence which have the same image). Also, we
sort the emoji in our emoji pickers alphabetically by shortcode, and so the
images for 🏢 and :office_building: show up next to each other, which
looks like a bug. This removes :office_building: as a shortcode, along with
several hundred other duplicates. It leaves some duplicates in that won't
give autocomplete or alphabetical ordering a problem, like (🚗,
:automobile:).
This fixes a regression introduced by our migration to track
subscribers for all public streams, where now users who are added to
an invite-only stream were receiving a mark_subscribed event
for a stream their browser didn't know existed, causing an exception.
To fix this, we now send a stream create event to the browser just
before the user receives the notification that it was added to the
invite-only stream.
The realm with string_id of "simple" just has three users
named alice, bob, and cindy for now. It is useful for testing
scenarios where realms don't have special zulip.com exception
handling.
In case realms have subdomains and the user hasn't been populated
yet in the Django User model, `ZulipLDAPAuthBackend` should not
rely on user's email domain to determine in which realm it should
be created in.
Fixes: #2227.
This fixes a bug where update_message_backend would do one memcached
query per user receiving a given message. Right now we just do a
single bulk database query, but in principle we could use
generic_bulk_cached_fetch to use the cache as well.
Apparently, we were comparing the full list of enabled authentication
methods (whether or not supported by the server) against the user's
selections among those supported by the server, which resulted in
authentication methods being always reported as different.
It turns out we were using malformed URLs in the image tags
(containing just a hostname, but no http(s)!) in what we were passing
to the Django templates for our digest/, which resulted in the Django
templates treating these URLs as http. Gmail recently cracked down on
loading images in HTTP, causing the emoji links to appear broken in
emails Zulip sends.
Fixes#3258.
This old helper has for years been used only by populate_db, and got
buggy (as of a recent refactoring). So we just call do_send_messages
directly instead.
Fixes the provisioning error we currently get in Travis CI.
This is a pretty minor change, but it makes it clear that we
have user_id in all the relevant states/events, so we might as
well use that for the check, since email is mutable and
slightly more difficult to reason about.
This changes bugdown to use the realm passed in by the caller (if any)
for rendering, fixing a problem where bots such as the notification
bot would have their messages rendering using the admin realm's
settings, not the settings of the realm their messages are being sent
into.
Also adds a test for the notification bot case.
Fixes#3215.
This moves the realm_filter_key variable, primarily used for clarity,
up from Bugdown into the render_markdown function.
We'll need this for the upcoming commits.
A lot of care has been taken to ensure we're using the realm that the
message is being sent into, not the realm of the sender, to correctly
handle the logic for cross-realm bot users such as the notifications
bot.
In order to correctly handle messages sent by cross-realm bots, we
need to specify the realm that the messages are being sent into in the
send message code path. The commit and its successors convert that
code path to include the realm the message is being sent to explicitly.
Before this commit, provisioning was done by executing provision.py,
which printed the log directly to stdout, making debugging harder.
This commit creates a wrapper bash script 'provision' in tools, which
calls 'zulip/scripts/tools/provision_vm.py' (the new location of
provision.py) and prints all the output to
'zulip/var/log/zulip/zulip_provision.log' via 'tee'.
Travis tests and docs have been modified accordingly.
Contributor visualization showing the avatar, user name and number
of commits for each contributors. The JSON data would be updated
upon deployment, triggered by the `update-prod-static` script.
Whether the emoji is valid is already being checked elsewhere, and
this duplicate regular expression makes it harder to understand what's
going on with Zulip's validation of emoji.
This should substantially improve the clarity of the code, since
inside bugdown, this is only being used as a hash key that happens to
usually be a realm ID, not used as a Realm ID.
This commit reverts commit "29673411df8dffd50198c0f01c5db8561a782adf"
due to the bug is not reproducible anymore (it seems likely this was a
bug in Django fixed between Django 1.8 and 1.10).
Fixes#1297.
- Change `stream_name` into `stream_id` on some API endpoints that use
`stream_name` in their URLs to prevent confusion of `views` selection.
For example:
If the stream name is "foo/members", the URL would be trigger
"^streams/(?P<stream_name>.*)/members$" and it would be confusing because
we intend to use the endpoint with "^streams/(?P<stream_name>.*)$" regex.
All stream-related endpoints now use stream id instead of stream name,
except for a single endpoint that lets you convert stream names to stream ids.
See https://github.com/zulip/zulip/issues/2930#issuecomment-269576231
- Add `get_stream_id()` method to Zulip API client, and change
`get_subscribers()` method to comply with the new stream API
(replace `stream_name` with `stream_id`).
Fixes#2930.
Remove events that don't exist.
Move handling issue events to separate function.
Make formatting strings using format function.
Change camelCase variable name convetion to using underscores.
Make unknown events error more clear.
Add issue_event_type_name param to all fixtures.
Previously, test_failed_signup_due_to_restricted_domain used a realm with
restricted domains, but also with invite_required = True. We didn't have a
test that tested for a failed signup in an open realm with restricted
domain, so edited test_failed_signup_due_to_restricted_domain to test for
that.
This reverts commit 7bf10ec74f.
Apparently, SockJS 1.1.1 is broken with the browser used in our legacy
desktop app, resulting in messages being silently not sent.
This adds some configuration options to settings.py, namely
PASSWORD_MIN_LENGTH and PASSWORD_MIN_QUALITY, which control
when the frontend validator invalidates the password.
Closes#2628
Previously, huddles were created outside of a transaction (and then
the actual initialization of the huddle object was done inside). This
made the huddle visible to other threads of execution while the
related data was still invisible due to transaction commit latency.
The general __init__ file is a more natural home, and where other endpoints
(e.g. create_realm, etc) live.
Also changes forms.ValidationError to django.core.exceptions.ValidationError
to match the rest of the file/codebase.
Bump up max length queries in `test_bulk_message_fetching()` to 11
in `zerver/tests/test_messages.py` to avoid test failing when run
this test alone.
Fixes#3087.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were
* Add a new function, get_realm_by_string_id.
* Change all calls to get_realm to use get_realm_by_string_id instead.
* Remove get_realm.
* (This commit) Rename get_realm_by_string_id to get_realm.
Part of a larger migration to remove the Realm.domain field entirely.
The new GitHub dispatcher integration was apparently totally broken,
because we hadn't tagged the new dispatcher endpoint as exempt from
CSRF checking. I'm not sure why the test suite didn't catch this.
- Add websocket client to create connection with SockJS websocket server.
It contains callback method to launch after connection setup.
- Add '--websocket' parameter to 'check_send_receive_time' script to
check websocket connection.
- Add testing websocket connection to production installation checking.
- Add cronjob to launch websocket connection nagios test.
This makes it possible for Zulip Nagios monitoring to check for
problems impacting the websockets sending code path, which is what all
web users use.
Update integration to use the latest Google API client.
Move Google Account authorization code to a separate file.
Move relevant files from 'bots/' to 'api/integrations/google/'.
Add documentation for integration.
Removes the dependence on postmonkey, which is a wrapper around
MailChimp API v1.3. MailChimp recommends using their v3.0 API directly,
rather than through a wrapper library.
Recent changes to the API removed Client.register(), and
this change restores the correct API call, although the
codepath this affects is probably ready for eventual
deprecation.
While this may not have created a clear user-facing bug, it seems less
confusing for do_invite_users to only create PreregistrationUser
objects for users who actually received an email invitation.
We only write domain to the session variable in one place,
accounts_home_with_domain, where we check that the domain is valid, that the
domain corresponds to an open realm, and that we are in the non-subdomains
case.
Previously, we were confusingly checking only a subset of the conditions
on reading back the domain in create_preregistration_user, and not checking
any of them when reading back the domain in get_realm_from_request.
Add test about avatar image was uploaded properly or not in
`tests.BotTest.test_add_bot_with_user_avatar` and
`tests.BotTest.test_patch_bot_avatar`.
Add `get_test_image_file()` and `avatar_disk_path()` in
`zerver.lib.test_helpers` and deduplicate some codes.
Fixes#1276.
- Markdow engine is not safe threading as global variable. In some cases
it generates previous markdown templates on concurent requests. It's
enough to generate markdown extensions per process to save time. Also
this solution is thread safe.
Previously, we included a special subscribe button in new stream
notifications, but that had 2 problems:
(1) The subscribe button would render badly if the stream was renamed.
(2) There wasn't an easy way to look at the stream when deciding
whether to subscribe.
This fixes the second problem, but not really the first.
This adds support for only allowing normal users with account age
equal or greater than a "waiting period" threshold to create streams;
this is useful for open organizations that want new members to
understand the community before creating streams.
If create_stream_by_admins_only setting is set to True, only admin users
were able to create streams. Now normal users with account age greater
or equal than waiting period threshold can also create streams.
Account age is defined as number of days passed since the user had
created his account.
Fixes: #2308.
Tweaked by tabbott to clean up the actual can_create_streams logic and
the tests.
First step in cleaning up populate_db.create_streams and
bulk_create.bulk_create_streams. Part of a series of commits to remove
Realm.domain from populate_db.
There is a change in Django 1.10 due to which whenever the password
of the user is changed the session hash changes. This change affects
us because we cache user profile objects and these cached objects need
to be refreshed. However, the signal sent by Django in which objects are
refreshed fails to refresh the cache for Tornado because it uses a
different cache prefix.
Note: Backend tests are not affected because they don't rely on Tornado.
Expose `is_mentioned` in `message` dict which contains
boolean value about our account is mentioned in the message
content or not.
This is already available via `flags`, but it seems worth making this
data point more explicit, given its importance in writing bots.
Fixes#2667.
Adding more additional information about user profile to
`zerver.views.pointer.get_profile_backend`, like `user_id`,
`full_name`, `email`, `is_bot`, `is_admin`, and `short_name` of the
user.
Zulip doesn't previously make use of the standard Django is_staff flag
(in that the Django admin site is disabled), but since conceptually
the /activity page would be part of the Django admin site if we were
using it (i.e. for server-level administrators), it makes sense to key
off of that rather than the previous, fragile, check for the realm
domain name.
Adding a reaction is now a PUT request to
/messages/<message_id>/emoji_reactions/<emoji_name>
Similarly, removing a reaction is now a DELETE request to
/messages/<message_id>/emoji_reactions/<emoji_name>
This commit changes the url and updates the views and tests.
This commit also adds a test for invalid emoji when removing reaction.
This includes making the default stream description setting into a
dict. That is an API change; we'll discuss it in the changelog but it
seems small enough to be OK.
With some small tweaks by tabbott to remove unnecessary backwards
compatibility code for the settings.
Fixes#2427.
This change adds support for displaying inline open graph previews for
links posted into Zulip.
It is designed to interact correctly with message editing.
This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.
By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.
Eventually, we may want to make this manageable via a (set of?)
per-realm settings. E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
This automates including of markdown files under
`templates/zerver/help/` to be tested by `test_public_urls` (in
zerver/tests/test_signup.py).
Fixes#2465.
This can be useful in scenarios where the network doesn't support
websockets. We don't include it in prod_settings_template.py since
it's a very rare setting to need.
Fixes#1528.
A few functions had arguments removed without having their type
annotations updated accordingly. As a result mypy version 0.4.6
thinks that the first type in the annotation is supposed to be
the type of `self`, a new mypy feature which we are not intending
to use here.
This commit adds support for removing reactions via DELETE requests to
the /reactions endpoint with parameters emoji_name and message_id.
The reaction is deleted from the database and a reaction event is sent
out with 'op' set to 'remove'.
Tests are added to check:
1. Removing a reaction that does not exist fails
2. When removing a reaction, the event payload and users are correct
The markdown files under templates/zerver/help/ are technically not
templates in the standard sense, and thus should not be being
checked with this code path.
(We probably do want to add a test to make sure they all render fine,
but that can be its own project.)
Modify backend test of create_streams_if_needed so that the newly
created streams have descriptions.
Modify casperjs test of filling out stream_creation_form so that
the newly created stream has a description.
Fixes: #2428.
- Add base tornado test case class.
- Add test for websocket connection.
- Add test for websocket authentication.
- Add test for sending private message with websocket.
- Add test for sending stream message with websocket.
Fixes#2230
This commit adds the following:
1. A reaction model that consists of a user, a message and an emoji that
are unique together (a user cannot react to a particular message more
than once with the same emoji)
2. A reaction event that looks like:
{
'type': 'reaction',
'op': 'add',
'message_id': 3,
'emoji_name': 'doge',
'user': {
'user_id': 1,
'email': 'hamlet@zulip.com',
'full_name': 'King Hamlet'
}
}
3. A new API endpoint, /reactions, that accepts POST requests to add a
reaction to a message
4. A migration to add the new model to the database
5. Tests that check that
(a) Invalid requests cannot be made
(b) The reaction event body contains all the info
(c) The reaction event is sent to the appropriate users
(d) Reacting more than once fails
It is still missing important features like removing emoji and
fetching them alongside messages.
Refactor list_to_streams and create_streams_if_needed to take a list
of dictionaries, instead of a list of stream names. This is
preparation for being able to pass additional arguments into the
stream creation process.
An important note: This removes a set of validation code from the
start of add_subscriptions_backend; doing so is correct because
list_to_streams has that same validation code already.
[with some tweaks by tabbott for clarity]
Previously, the key prefix was based on the process id due to which
the JS tests couldn't properly flush user profiles from the cache as
our application spans over multiple processes. This problem becomes
apparent when in json_change_settings view after changing the user_profile
the tornado views continue to get the cached user profile corresponding
to their process id.
Clean up the instances of self.assertIn("string", result.content.decode("utf-8")),
and replace them with self.assert_in_response("string").
Fixes: #2313
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.
If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.
With some tweaks to the output from tabbott.
Fixes#1441.
Previously, we rejected the HEAD requests that the trello integration
uses to check if the server accepts the integration.
Add decorator for returning 200 status code if request is HEAD.
Fixes: #2311.
Django reverts all the changes after running a test but the
client cache retained the deleted value, this caused the
subsequent tests to fail due to invalid foreign key constraints.
This commit fixes the issue by prefixing the cache name in
client cache with KEY_PREFIX which is bounced after every test.
Tomasz Kolek
Harshit Bansal
Tim Abbott
Rishi Gupta
Yago González
avisrivastava254084
Raghav Jajodia
Steve Howell
Adarsh S
Rohith Asrk
Sampriti Panda
sinwar
Tommy Ip
Umair Khan
Robert Hönig
Bojidar Marinov
Rafid Aslam
JefftheBest1
Brock Whittaker
Jackson
Vamshi Balanaga
anirudhjain75
K.Kanakhin
Prabod Rathnayaka
Juan Verhook
paxapy
Igor Tokarev
vaibhav
Kracekumar R
reyha
reallyly
Jason Le
Zac Pullar-Strecker
Arpith Siromoney
Tomonori Murakami
Joy Chen
root
Bickio
Mikebarson
sidhant bhavnani
bulat22101
Noel Tautges
AZtheAsian
Reid Barton
nikolay
Sidhant Bhavnani
Alex Huang
sylvan1
Calvin Lee
Anders Kaseorg
Vladislav Manchev
hackerkid
Pweaver (Paul Weaver)
Tim Abbott
Tomasz Kolek