Server settings should just be added to the context in build_email, so that
the individual email pathways (and later, the email testing framework)
doesn't have to worry about it.
Realm.notifications_stream is not a boolean, Text or integer field, and
thus doesn't fit into the do_set_realm_property framework. Added function
to update it in actions.py. Altered the view, realm.py, to accept
stream-id. Also, notifications stream can be disabled by sending a
negative id.
Rationale: For the more off-to-the-side edit history view, changes
are easier to digest by highlighting deleted content in red followed
immediately by added and changed content in green.
TODO: Toggle for showing the edited messages without highlighting;
deleted content would not be shown in this view.
Add 'Type of bot' option for bots by adding dropdown option in
settings->"Your bots". For now, this allows creating incoming webhook
bots in addition to default bots.
This will enable users to add a bot as an incoming webhook
(in addition to add full-featured bots).
With various minor tweaks and cleanups by tabbott.
Fixes#2186.
This page describes software the user will get from upstream for
their own devices, independent of what's on the server they're
using. So it should live in a place maintained together with
that other software, rather than be distributed and versioned
with the server.
The use of ZILENCER_ENABLED to tell the difference is rather a hack
but is currently how we do this in the small handful of similar
spots; see #5245.
Fixes#5234.
When the last user on a private stream is removed, the stream is no
longer possible to administer, and thus should be marked as
deactivated, so that default streams entries are removed and it no
longer appears in the UI as a non-administerable broken stream.
The `data-toggle` property prevented the new style of overlay modals
from launching, and regardless, isn't a future-proof options for how
this should work.
Previously, we were incorrectly using the get_unique_open_realm
function to determine whether we're in the (common) single-realm
server case and should just display an org-info-enabled login form on
the homepage.
Now, we use a slightly different function extracted from
get_unique_open_realm that doesn't check whether the realm is
invite-only.
Fixes#4841.
This is CVE-2017-0896.
Apparently, this setting never actually was wired up to anything other
than hiding the UI widget.
Huge thanks to Ibram Marzouk from the HackerOne community for finding
this security bug.
This test fails on self.assertTrue(delay < 0.001 * num_ids, error_msg)
randomly. This commit adds debug code to see what the real values of
paramters are.
To get accurate count of the queries, we should make sure that
caches don't come into play. If we count queries while caches are
filled, we will get a lower count. Caches are not supposed to be
persistent, so our test can also fail if cache is invalidated
during the course of the unit test.
This commit solves the problem with Stream cache. This cache comes
into play when we use `get_stream` function. If cache is valid,
we will not issue queries to Stream and Recipient table. I think
the problem was one of those rare occasions when the Stream cache
got invalidated during the course of the test, due to which query
count was increased by 2. After this commit, we intentially invalidate
the Stream cache.
This makes it possible for Zulip administrators to delete messages.
This is primarily intended for use in deleting early test messages,
but it can solve other problems as well.
Later we'll want to play with the permissions model for this, but for
now, the goal is just to integrate the feature.
Note that it saves the deleted messages for some time using the same
approach as Zulip's message retention policy feature.
Fixes#135.
Previously, all notification preference setting had a dedicated test
and setter. Now, all are handled through a modular function using the
property_types framework.
mypy will error because of the attribute "request" on the LogRecord
object. Since this field is added in our tests dynamically and is not
on the base object, for now we will ignore the type.
When we create a stream, we usually send a welcome message on the
stream itself as well as an announcement on the announcement stream,
but we no longer PM the individual users. Hopefully this will be
more pleasant for users (less spammy), and it also will make creating a
stream a lot faster.
We still send notifications when we add subscribers to an existing
stream.
We now pre-populate the streams in DEFAULT_NEW_REALM_STREAMS
(social/general/zulip, unless somebody changes settings.py) with
welcome messages. This makes the streams appear to be active
right away, and it also gives the Zulip realm less of a
blank-slate feeling when you create it.
This change only affects the normal web-based create-realm flow.
It doesn't impact the management commands for creating realms
or setting default streams.
This fixes 2 issues:
* The term "@-mentioned" is simplified to "mentioned".
* We would incorrectly list other people who sent context messages as
among the people who mentioned you.
Now, in the event of messages between two other members of a huddle,
the missed message emails are threaded in "Group PMs with name1 and
name2" and not in separate threads by sender.
Also, now the order of recipients in get_display_recipient consistent
with the order of names that appears in the list of personal messages
on the left sidebar.
Fixes most of #4553.
These handlers will kick into action when is_signup is False. In case
the account exists, the user will be logged in, otherwise, user will
be asked if they want to proceed to registration.
This is one of the last major endpoints that were still done in the
pre-REST style.
While we're at it, we change the endpoint to expect a stream ID, not a
stream name.
This fixes most cases where we were assigning a user to
the var email and then calling get_user_profile_by_email with
that var.
(This was fixed mostly with a script.)
The example_user() function is specifically designed for
AARON, hamlet, cordelia, and friends, and it allows a concise
way of using their built-in user profiles. Eventually, the
widespread use of example_user() should help us with refactorings
such as moving the tests users out of the "zulip.com" realm
and deprecating get_user_profile_by_email.
On slower systems or virtual environments, when you are running
tests in parallel mode, sometimes the time taken to send messages
in this test exceeds 1 second which resets the limit.
We used to use constructions like
from_email = "Zulip <%s>" % (settings.NOREPLY_EMAIL_ADDRESS,)
but no longer do. All references to settings.NOREPLY_EMAIL_ADDRESS in the
codebase now do not append a display name.
This commit is a step towards the goal of replacing most of the
send_future_email pathway with a call to send_email.
Note that this commit changes the default value of sender from "Zulip
<NOREPLY_EMAIL_ADDRESS>" to "NOREPLY_EMAIL_ADDRESS". NOREPLY_EMAIL_ADDRESS
will soon be changed to have the Zulip in front.
Note that the correctness of this commit relies on the fact that
send_future_email also sets the sender to settings.NOREPLY_EMAIL_ADDRESS by
default (in the body of the function).
Fixes regression introduced in 326f9a85. The test indirectly makes a call to
email_is_not_mit_mailing_list, which then calls
DNS.dnslookup("%s.pobox.ns.athena.mit.edu" % username, DNS.Type.TXT).
If a user is trying to register for a mit zephyr mirroring realm, we send
them a specific registration email with a link to a few more instructions.
There is only one server that we know about that has such a realm, and that
server uses subdomains. This commit changes the logic to work in the
subdomains case, rather than in the non-subdomains case (though see next
para).
Note that the current check is deceptive, and is not actually correct in the
non-subdomains case. The prereg user has a realm only in the atypical case
of someone registering via the special URL for completely-open realms.
To do this correctly in the non-subdomains case, we would need to copy a
bunch of the logic from the beginning of accounts_register to figure out
which realm the user is signing up for, so that we can check if that realm
is a zephyr mirroring realm. Given how complicated the registration code is
already, I think it is probably not worth it at the moment. This commit also
removes the partial (deceptive) check, since I think it does more harm than
good.
Specifically, this makes easily available to the desktop and mobile
apps data on the server's configuration, including important details
like the realm icon, name, and description.
It deprecates /api/v1/get_auth_backends.
Previously, api_key_only_webhook_view passed 3 positional arguments
(request, user_profile, and client) into a function. However, most
of our other auth decorators only pass 2 positional arguments. For
the sake of consistency, we now make api_key_only_webhook_view set
request.client and pass only request and user_profile as positional
arguments.
This is a better solution to the problem of how _pg_re_escape should
handle the null character. There's really no good reason to have a
null character in a stream name.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.
Fixes#1134
This makes it possible for the Zulip mobile apps to use the normal web
authentication/Oauth flows, so that they can support GitHub, Google,
and other authentication methods we support on the backend, without
needing to write significant custom mobile-app-side code for each
authentication backend.
This PR only provides support for Google auth; a bit more refactoring
would be needed to support this for the GitHub/Social backends.
Modified by tabbott to use the mobile_auth_otp library to protect the
API key.
We'll need to implement a version of the simple decoding/decryption
logic used by this library in the mobile code as well, but that should
be simple enough.
This completes a major redesign of the Zulip login and registration
pages, making them look much more slick and modern.
Major features include:
* Display of the realm name, description and icon on the login page
and registration pages in the subdomains case.
* Much slicker looking buttons and input fields.
* A new overall style for the exterior of these portico pages.
This new feature makes it possible to request a different set of
initial data from the event_types an API client is subscribing to.
Primarily useful for mobile apps, where bandwidth constraints might
mean one wants to subscribe to events for a broader set of data than
is initially fetched, and plan to fetch the current state in future
requests.
- Add aggregated info to real-time updated presence status.
- Update `presence events` test case with adding aggregated
information to presence event.
- Add test case for updating presence status for user which
send state from multiple clients.
Fixes#4282.
This fixes a confusing issue where a user might try resetting the
password for an email account that in part of a different Zulip
organization.
Is a useful early step towards making Zulip support reusing an email
in multiple realms.
Fixes: #4557.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server. Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.
- Add script wrapper to communicate postfix pipe with django web server
over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
based on updated script.
- Add tests.
Tweaked by tabbott to adjust the directory and set better defaults.
Fixes#2421.
Rename 'zulip_internal' decorator to 'require_server_admin', add
documentation for 'server_admin', explaining how to give permission
for ./activity page.
Fixes: #1463.
This is basically just using the new check_dict_only everywhere, with
a few exceptions:
* New self.check_events_dict automatically adds the id field to avoid
duplicating it ~80 times.
* Set log=False for many of the testing action functions to remove the
timestamp field from their returned event dictionaries, since it's
not needed and is the result of a deprecated log_event function.
Wasn't sure if the subscription_field list in do_test_subscribe_events
could contain optional arguments, so I left the call to check_dict on
along with a TODO.
Fixes: #1370.
In this commit we add a logout wrapper so as to enable developers
to just do self.logout instead of doing a post request at API
endpoint for logout. This is achieved by adding a wrapper function
for the Django's client.logout contained in TestCase. We add this
by extending ZulipTestCase to have a logout function.
This is an incomplete cleaned-up continuation of Lisa Neigut's push
notification bouncer work. It supports registration and
deregistration of individual push tokens with a central push
notification bouncer server.
It still is missing a few things before we can complete this effort:
* A registration form for server admins to configure their server for
this service, with tests.
* Code (and tests) for actually bouncing the notifications.
In cases where old unread messages in the home view might have been
leaked (either due to bugs or unusual muting interactions), it's
theoretically possible for the first unread message in the home view
to be far older than the pointer.
Since the Zulip mobile app is loading messages following the
use_first_unread logic, we need to plug this gap.
Probably a longer-term solution will involve changing how
update_message_flags works to automatically advance the pointer, but
this change should make it possible for the mobile apps to
consistently use the `use_first_unread` mechanism for fetching the
latest home view messages.
With tweaks to the tests by tabbott.
Fixeszulip/zulip-mobile#422.
The previous logic was that anyone with a link to a file could send it
to other users, but only the owner could make a file realm-public.
This had some confusing corner cases.
The new logic is much simpler:
* Only the file's owner/uploader can include a file in a message for
the first time.
* Anyone with access to read a file can share it with others by
including it in messages they send.
* Once a file has been sent to a public stream, any user in the realm
can access it.
In this commit we fix the occasionally breaking tests for
test_home.HomeTest.test_bad_narrow which were the result of
us patching global settings in test_upload to add some new emails
to CROSS_REALM_BOT_EMAILS and not rolling back.
textsearch based full text search doesn't match text in link tag but
PGroonga based full text search can match text in link tag.
Without this change, highlighting text in link tag generates broken
HTML.
This commit makes sure that GitHubAuthBackend will only authenticate
using its own authenticate method. This is done by adding a new
Python Social Auth strategy which instead of calling authenticate
method of Django, calls the authenticate of the backend directly.
The problem this commit solves is that while authenticating through
GitHub backend, we were ending up getting authenticated through
ZulipDummyBackend. This might happen because the default strategy used
by Python Social Auth calls the authenticate method of Django which
iterates over all the backends and tries the authenticate methods
which match with the function arguments. The new strategy this commit
adds calls the authenticate method of GitHub backend directly which
makes sense because we already know that we want to authenticate with
GithHub.
The actual problem of why we are ending up on ZulipDummyBackend is
still a mystery because the function arguments passed to its
authenticate method are different. It shouldn't be called.
Earlier, a stack was being used to go through the message and search
for links. Because of this, in some cases the images were added to
the preview in reverse. Using a queue will keep the image previews in
the same order as they appeared in the message.
Fixes#4453.
This fixes an issue with a nondeterministic number of database queries
being used in fetching bulk messages from the database. The source of
the problem was that we were fetching _all_ messages, not just the 600
that had been created by the test, and thus if the set of streams
present in messages in the test fixtures (which is random) changes,
the number of streams used (and thus number of queries) would change.
Show a user friendly message to the user if email is invalid.
Currently we show a generic message:
"Your username or password is incorrect."
The only backend which can accept a non-email username is LDAP.
So we check if it is enabled before showing the custom message.
This removes individual tests for realm properties and replaces them
with a generic do_set_realm_property_test function to test each
property in the Realm.property_types attribute.
Addresses part of #3854.
This replaces individual tests for realm properties with a generic
do_test_realm_update_api function to test each property in the
Realm.property_types attribute.
Addresses part of #3854.
This is a remerge of e985b57259 (after
resolving merge conflicts, updating the tests, adding mypy annotations
etc.), which should now be correct, because we've done the necessary
database migration.
The rebase/remerge work was done by Tim Abbott and Aditya Bansal.
This is an important part of #320.
Django uses arguments to differentiate between different authenticate
function so it is important to pass arguments in a predictable manner.
Keyword args will test the name of the argument as well.
The web app doesn't need any presence data for its first ping to
the server, because it already has up-to-date presence info in
page_params. So now we can tell the server not to send us a big
payload that we were already ignoring.
Due to Pgroonga regression, there is a difference in search
result between Travis and development env due to which one of
our tests fails. This commit makes sure that the test passes
for both strings till the Pgroonga bug is resolved.
This commit adds the backend support for a new style of tutorial which
allows for highlighting of multiple areas of the page with hotspots that
disappear when clicked by the user.
Modify `bot_owner_user_ids()` to return the user_ids of only
admins and bot owners instead of all the current active users.
This was causing a traceback on the frontend.
Fixes: #3391.
- Add message retention period field to organization settings form.
- Add css for retention period field.
- Add convertor to not negative int or to None.
- Add retention period setting processing to back-end.
- Fix tests.
Modified by tabbott to hide the setting, since it doesn't work yet.
The goal of merging this setting code now is to avoid unnecessary
merge conflicts in the future.
Part of #106.
This adds helpful email notifications for users who just logged into a
Zulip server, as a security protection against accounts being hacked.
Text tweaked by tabbott.
Fixes#2182.
This fixes a leak of this setting change that resulted from the
unusual way that our Tornado system sets this variable early in the
management command.
Fixes#3685.
This fixes 2 issues:
* Being added to an invite_only stream did not correctly update the
"streams" key of the initial state.
* Once that's resolved, subscribe_to_stream when called on a
nonexistant stream would both send a "create" event (from
create_stream_if_needed) and an "occupy" event (from
bulk_add_subscriptions).
The second event should just be suppressed in that case, and this
implements that suppression.
We previously didn't apply the default language event change
correctly.
Not super important as a bug, since we require the user to reload the
browser for their changes to take effect, but this will save time if
we ever change that.
zerver/lib/actions: removed do_set_realm_* functions and added
do_set_realm_property, which takes in a realm object and the name and
value of an attribute to update on that realm.
zerver/tests/test_events.py: refactored realm tests with
do_set_realm_property.
Kept the do_set_realm_authentication_methods and
do_set_realm_message_editing functions because their function
signatures are different.
Addresses part of issue #3854.
This makes get_stream match get_realm, get_user_profile_by_email,
etc., in interface, and is more convenient for mypy annotations
because `get_stream` now doesn't return an Optional[Stream].
We use the same strategy Zulip already uses for starred messages,
namely, creating a new UserMessage row with the "historical" flag set
(which basically means Zulip can ignore this row for most purposes
that use UserMessage rows). The historical flag is ignored, however,
in determining which users' browsers to notify about new reactions,
and thus the user will get to see the reaction appear when they click
a message (and any reactions other users later add, as well!).
There's still something of a race here, in that if some users react to
a message while the user is looking at the unsubscribed stream but
before the user reacts to that message, those reactions will not be
displayed to that user (so counts will be a bit lower, or something).
This race feels small enough to ignore for now.
Fixes#3345.
In aa880b0419, we used the raw
do_set_realm_description method rather than calling the API, which
meant that the API success path wasn't actually tested.
This adds an organization description field to the Realm model, as well as
an input field to the organization settings template. Added three tests.
Set the max length of the field to 100 characters.
Fixes#3962.
Changing assert_in_success_response to require List[Text] instead of
Iterable[Text] prevents the following misuse:
self.assert_in_response_success("message", response)
Currently, this will check whether 'm', 'e', 's', 'a', and 'g' separately
appear in the response, which is probably not the intended behavior. The
correct usage is as follows:
self.assert_in_response_success(["message"], response)
This of course only works in the 2 minute window where missed-message
emails are planned, but nonetheless likely avoids common cases of
emailing users with deleted messages.
Fixes: #3873.
Missed-message email replies using the reply-to of
noreply@zulipchat.com shouldn't advertise that "just replying" will
work.
Rebased and commit message rewritten by tabbott.
Fixes#3965.
On reloading the page after disabling email changes does not check
the "Prevent users from changing their email address".
Adding realm_email_changes_disabled to page_params_core_fields fixes the problem.
This fixes 2 related issues:
* We incorrectly would report authentication methods that are
supported by a server (but have been disabled for a given
realm/subdomain) as supported.
* We did not return an error with an invalid subdomain on a valid
Zulip server.
* We did not return an error when requesting auth backends for the
homepage if SUBDOMAINS_HOMEPAGE is set.
Comes with complete tests.
Our linter for translation strings shouldn't check test files, since
then we'll end up translating non-user-facing strings.
So we fix that, and actually add the opposite lint rule.
Change applies to both subdomains and non-subdomains case, though we use
just the EXTERNAL_HOST in the non-subdomains case if there is only 1 realm.
Fixes#3903.
- Add settings parameter for max realm icon size.
- Add settings parameter for max user avatar size.
- Add checking file size to avatar and icon
uploading views.
- Transfer file size limit parameter to frontend.
- Add tests.
- Add `OFFLINE_THRESHOLD_SECS` settings parameter
to handle offline period.
- Set aggregated status to offline if user's status
haven't changed for `OFFLINE_THRESHOLD_SECS` period.
- Add test for offline aggregated status.
- Add aggregated status to user presence status dict.
- Add tests for aggregated presence status.
- Fix removing unused keys from status dict
with aggregated data for user.
Fixes#3692
- Add new 'missedmessage_email_senders' queue for sending missed messages emails.
- Add the new worker to process 'missedmessage_email_senders' queue.
- Split aggregation missed messages and sending missed messages email
to separate queue workers.
- Adapt tests for sending missed emails to the new logic.
Fixes#2607
This refactoring is preparation for being able to forward push
notifications to users on behalf of another Zulip server.
The goal is to remove access to the current server's database from the
send_*_push_notification code paths.
This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10. Now that we're on 1.10,
it's no longer required.
This currently only supports this in emoji reactions, not in actual
emoji in message bodies, but it's a great start for people who want a
text-only view.
Tweaked to update the text by tabbott.
Fixes#3169.
datetime.utcnow() is a timezone-naive datetime. The Django ORM interprets it
in the settings.TIME_ZONE timezone (e.g. 'America/New_York' in the
development server). We perhaps haven't noticed errors yet since with
'America/New_York' all it means is that emails are sent 5 hours early, or a
slightly different set of messages are included in the digest.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways. First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte. We fix this and add tests.
This completes the effort to get zerver/views/messages.py to 100%
test coverage.
Fixes#1006.
When you edit a message to contain links, and URL previews are
enabled, previously we'd throw an exception, because the realm ID
wasn't included in the event.
Also adds a test so that we can have effective test coverage on this
codepath, though this history is actually that I found the bug through
writing this test :).
Rishi Gupta
Aditya Ramesh
Steve Howell
Tim Abbott
Akhil
Jack Zhang
Abhijeet Kaur
Greg Price
Aditya Bansal
Tommy Ip
Umair Khan
K.Kanakhin
Harshit Bansal
vaibhav
Maxim Averin
Ron Shafii
Ethan Estrada
Reid Barton
Elliott Jin
Vishnu Ks
Andrew Archer
Rick Chern
umkay
Yago González
Andrew Archer
kb0rg
Lech Kaiel
Matt Long
Lukasz Prasol
James Rowan
Cory Lynch
andrewallen00
kunall17
Rohitt Vashishtha
Eeshan Garg
sinwar
fionabunny
Raghav Jajodia
Neeraj Wahi
Brock Whittaker
Michael
Jacob Hurwitz
Kouhei Sutou
Sampriti Panda
Sarah
rahuldeve
Tomasz Kolek
Amy Liu
hollywoodno
Akash Kothawale
Ayush Jain
Steven Ganz
Susan Salituro
dattatreya303
Philip Skomorokhov
Sourav Badami
Igor Tokarev