This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization. It controls access through 2
key mechanisms:
* For long-ago edited messages, get_messages removes the edit history
content from messages it sends to clients.
* For newly edited messages, clients are responsible for checking the
setting and not saving the edit history data. Since the webapp was
the only client displaying it before this change, this just required
some changes in message_events.js.
Significantly modified by tabbott to fix some logic bugs and add a
test.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).
When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779. We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
We are deprecating local_id/local_message_id on the Python server.
Instead of the server knowing about the client's implementation of
local id, with the message id = 9999.01 scheme, we just send the
server an opaque id to send back to us.
This commit changes the name from local_id -> client_message_id,
but it doesn't change the actual values passed yet.
The goal for client_key in future commits will be to:
* Have it for all messages, not just locally rendered messages
* Not have it overlap with server-side message ids.
The history behind local_id having numbers like 9999.01 is that
they are actually interim message ids and the numerical value is
used for rendering the message list when we do client-side rendering.
Prior to this commit, 7 megabytes of images (through 253 individual requests)
were heavily slowing down the initial load. With this commit, we load only the
logos (60 or so images).
Documentation and images for the individual integration sub-pages is requested
separately using the /integrations/doc/ endpoint, which returns HTML.
Use bool_change if the user_display setting property_type is bool, so that no additional code needs to be added to test_events for new boolean user display settings.
The SubscriptionAPITest class variables `realm` and `test_realm` stores
the same information and are redundant. I have eliminated all occurances
of self.realm and replaced with self.test_realm.
We were using relative URLs for realm emojis in missed message emails.
Since the email server is not same as the Zulip server and doesn't
have the realm emoji files, they were rendered as broken images. This
commit fixes them to use absolute URL.
Fixes: #5692.
The MitUser table was removed in df525ad.
confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
templates.insert(0, 'confirmation/confirm_%s.html'
% (obj._meta.model_name,))
The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
The Realm.DoesNotExist exception will never be raised if we pass a
string_id as in that case zerver.models.get_realm function would be
used for fetching. realm.zerver.models.get_realm uses filter query
so the exception will not be raised even if there are no realm which
matches the query.
Tests added by tabbott.
This system hasn't been in active use for several years, and had some
problems with it's design. So it makes sense to just remove it to declutter
the codebase.
Fixes#5655.
The change password form http://localhost:9991/#settings/your-account
don't have data-min-length and data-min-quality attributes. The
account_settings.handlebar which has the change password form is
rendered client side. So we have to pass the value of min length
and quality in page params to set the data-min-length and
data-min-quality attributes.
This new library is intended to make it easy for management commands
to access a realm or a user in a realm without having to duplicate any
of the annoying parsing/extraction code.
This fixes a bug where we would previously not validate the format of
APNS tokens before writing them to the database, which could lead to
exceptions in the push notifications system if a buggy mobile app
submitted invalid format tokens.
The refer_friend endpoint is about to be deprecated, so we
test with invite_users instead, and I tried to make the
dummy value returned by the mocked gettext function a little
more clear.
Instead of removing an emoji from the database, just mark them as
deactivated so that they can't be used further but can be rendered
properly in reactions and messages.
Fixes: #4750.
No change in behavior.
Also makes the first step towards converting all uses of
settings.ZULIP_ADMINISTRATOR and settings.NOREPLY_EMAIL_ADDRESS to
FromAddress.*.
Once everything is converted, it will be easier to ensure that future
development doesn't break backwards compatibility with the old style of
settings emails.
This will allow for customized senders for emails, e.g. 'Zulip Digest' for
digest emails and 'Zulip Missed Messages' for missed message emails.
Also:
* Converts the sender name to always be "Zulip", if the from_email used to
be settings.NOREPLY_EMAIL_ADDRESS or settings.ZULIP_ADMINISTRATOR.
* Changes the default value of settings.NOREPLY_EMAIL_ADDRESS in the
prod_setting_template to no longer have a display name. The only use of
that display name was in the email pathway.
Previously, we were checking for "Zulip" entries in User-Agent strings
at the very end of the list, not the beginning. A Zulip User-Agent
should have priority, since it includes the User-Agent strings for a
lot of other browsers, and also it will always be the most specific
value.
Unicode emojis when rendered should display canonical short name.
Similarly, the alt text should be of the format `:<short_name>:`.
For both of these we currently display the actual unicode symbol.
As some systems don't have the fonts necessary for displaying them
properly, they are rendered as empty square blocks. This commit also
ensures that the markup generated for emoji generated by canonical
name and by an unicode emoji is same.
Fixes: #5555.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
Currently we only pass headers in the first client_get call but
sometimes the effective request which reaches the view is through
a later call to the client_get in this function. Due to which
headers are not passed.
This isn't very slick, but it should get the main points down,
and it's past time we got something like this up. Definitely
needs in the future another pass at the text, and also some images
(screenshots, etc.) and styling.
We used shortnames for mentioning users before we had autocomplete
feature. Since we now have autocomplete typeahead, this syntax is
no more useful and just causes problems. This commit removes the
shortname mention syntax.
Fixes: #4189.
Previously, the only required field in RegistrationForm was the full
name (and possibly ToS, depending on settings). This meant that if
LDAP was configured, realm creation would break, because the form
would be valid the first time one landed on it, before the user even
filled it out!
The correct fix is to make the extra fields required in
RegistrationForm in the event that we're doing realm creation.
It's possible that a cleaner fix would be to use a subclass.
With a test from Umair Waheed Khan.
Fixes#5387.
A deactivated realm emoji should neither be accepted further as a
reaction nor its further occurences in a message be rendered as an
emoji. However, all the old occurences should continue to render
normally.
Server settings should just be added to the context in build_email, so that
the individual email pathways (and later, the email testing framework)
doesn't have to worry about it.
Realm.notifications_stream is not a boolean, Text or integer field, and
thus doesn't fit into the do_set_realm_property framework. Added function
to update it in actions.py. Altered the view, realm.py, to accept
stream-id. Also, notifications stream can be disabled by sending a
negative id.
Rationale: For the more off-to-the-side edit history view, changes
are easier to digest by highlighting deleted content in red followed
immediately by added and changed content in green.
TODO: Toggle for showing the edited messages without highlighting;
deleted content would not be shown in this view.
Add 'Type of bot' option for bots by adding dropdown option in
settings->"Your bots". For now, this allows creating incoming webhook
bots in addition to default bots.
This will enable users to add a bot as an incoming webhook
(in addition to add full-featured bots).
With various minor tweaks and cleanups by tabbott.
Fixes#2186.
This page describes software the user will get from upstream for
their own devices, independent of what's on the server they're
using. So it should live in a place maintained together with
that other software, rather than be distributed and versioned
with the server.
The use of ZILENCER_ENABLED to tell the difference is rather a hack
but is currently how we do this in the small handful of similar
spots; see #5245.
Fixes#5234.
When the last user on a private stream is removed, the stream is no
longer possible to administer, and thus should be marked as
deactivated, so that default streams entries are removed and it no
longer appears in the UI as a non-administerable broken stream.
The `data-toggle` property prevented the new style of overlay modals
from launching, and regardless, isn't a future-proof options for how
this should work.
Previously, we were incorrectly using the get_unique_open_realm
function to determine whether we're in the (common) single-realm
server case and should just display an org-info-enabled login form on
the homepage.
Now, we use a slightly different function extracted from
get_unique_open_realm that doesn't check whether the realm is
invite-only.
Fixes#4841.
This is CVE-2017-0896.
Apparently, this setting never actually was wired up to anything other
than hiding the UI widget.
Huge thanks to Ibram Marzouk from the HackerOne community for finding
this security bug.
This test fails on self.assertTrue(delay < 0.001 * num_ids, error_msg)
randomly. This commit adds debug code to see what the real values of
paramters are.
To get accurate count of the queries, we should make sure that
caches don't come into play. If we count queries while caches are
filled, we will get a lower count. Caches are not supposed to be
persistent, so our test can also fail if cache is invalidated
during the course of the unit test.
This commit solves the problem with Stream cache. This cache comes
into play when we use `get_stream` function. If cache is valid,
we will not issue queries to Stream and Recipient table. I think
the problem was one of those rare occasions when the Stream cache
got invalidated during the course of the test, due to which query
count was increased by 2. After this commit, we intentially invalidate
the Stream cache.
This makes it possible for Zulip administrators to delete messages.
This is primarily intended for use in deleting early test messages,
but it can solve other problems as well.
Later we'll want to play with the permissions model for this, but for
now, the goal is just to integrate the feature.
Note that it saves the deleted messages for some time using the same
approach as Zulip's message retention policy feature.
Fixes#135.
Previously, all notification preference setting had a dedicated test
and setter. Now, all are handled through a modular function using the
property_types framework.
mypy will error because of the attribute "request" on the LogRecord
object. Since this field is added in our tests dynamically and is not
on the base object, for now we will ignore the type.
When we create a stream, we usually send a welcome message on the
stream itself as well as an announcement on the announcement stream,
but we no longer PM the individual users. Hopefully this will be
more pleasant for users (less spammy), and it also will make creating a
stream a lot faster.
We still send notifications when we add subscribers to an existing
stream.
We now pre-populate the streams in DEFAULT_NEW_REALM_STREAMS
(social/general/zulip, unless somebody changes settings.py) with
welcome messages. This makes the streams appear to be active
right away, and it also gives the Zulip realm less of a
blank-slate feeling when you create it.
This change only affects the normal web-based create-realm flow.
It doesn't impact the management commands for creating realms
or setting default streams.
This fixes 2 issues:
* The term "@-mentioned" is simplified to "mentioned".
* We would incorrectly list other people who sent context messages as
among the people who mentioned you.
Now, in the event of messages between two other members of a huddle,
the missed message emails are threaded in "Group PMs with name1 and
name2" and not in separate threads by sender.
Also, now the order of recipients in get_display_recipient consistent
with the order of names that appears in the list of personal messages
on the left sidebar.
Fixes most of #4553.
These handlers will kick into action when is_signup is False. In case
the account exists, the user will be logged in, otherwise, user will
be asked if they want to proceed to registration.
This is one of the last major endpoints that were still done in the
pre-REST style.
While we're at it, we change the endpoint to expect a stream ID, not a
stream name.
This fixes most cases where we were assigning a user to
the var email and then calling get_user_profile_by_email with
that var.
(This was fixed mostly with a script.)
The example_user() function is specifically designed for
AARON, hamlet, cordelia, and friends, and it allows a concise
way of using their built-in user profiles. Eventually, the
widespread use of example_user() should help us with refactorings
such as moving the tests users out of the "zulip.com" realm
and deprecating get_user_profile_by_email.
On slower systems or virtual environments, when you are running
tests in parallel mode, sometimes the time taken to send messages
in this test exceeds 1 second which resets the limit.
We used to use constructions like
from_email = "Zulip <%s>" % (settings.NOREPLY_EMAIL_ADDRESS,)
but no longer do. All references to settings.NOREPLY_EMAIL_ADDRESS in the
codebase now do not append a display name.
This commit is a step towards the goal of replacing most of the
send_future_email pathway with a call to send_email.
Note that this commit changes the default value of sender from "Zulip
<NOREPLY_EMAIL_ADDRESS>" to "NOREPLY_EMAIL_ADDRESS". NOREPLY_EMAIL_ADDRESS
will soon be changed to have the Zulip in front.
Note that the correctness of this commit relies on the fact that
send_future_email also sets the sender to settings.NOREPLY_EMAIL_ADDRESS by
default (in the body of the function).
Fixes regression introduced in 326f9a85. The test indirectly makes a call to
email_is_not_mit_mailing_list, which then calls
DNS.dnslookup("%s.pobox.ns.athena.mit.edu" % username, DNS.Type.TXT).
If a user is trying to register for a mit zephyr mirroring realm, we send
them a specific registration email with a link to a few more instructions.
There is only one server that we know about that has such a realm, and that
server uses subdomains. This commit changes the logic to work in the
subdomains case, rather than in the non-subdomains case (though see next
para).
Note that the current check is deceptive, and is not actually correct in the
non-subdomains case. The prereg user has a realm only in the atypical case
of someone registering via the special URL for completely-open realms.
To do this correctly in the non-subdomains case, we would need to copy a
bunch of the logic from the beginning of accounts_register to figure out
which realm the user is signing up for, so that we can check if that realm
is a zephyr mirroring realm. Given how complicated the registration code is
already, I think it is probably not worth it at the moment. This commit also
removes the partial (deceptive) check, since I think it does more harm than
good.
Specifically, this makes easily available to the desktop and mobile
apps data on the server's configuration, including important details
like the realm icon, name, and description.
It deprecates /api/v1/get_auth_backends.
Previously, api_key_only_webhook_view passed 3 positional arguments
(request, user_profile, and client) into a function. However, most
of our other auth decorators only pass 2 positional arguments. For
the sake of consistency, we now make api_key_only_webhook_view set
request.client and pass only request and user_profile as positional
arguments.
This is a better solution to the problem of how _pg_re_escape should
handle the null character. There's really no good reason to have a
null character in a stream name.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.
Fixes#1134
This makes it possible for the Zulip mobile apps to use the normal web
authentication/Oauth flows, so that they can support GitHub, Google,
and other authentication methods we support on the backend, without
needing to write significant custom mobile-app-side code for each
authentication backend.
This PR only provides support for Google auth; a bit more refactoring
would be needed to support this for the GitHub/Social backends.
Modified by tabbott to use the mobile_auth_otp library to protect the
API key.
We'll need to implement a version of the simple decoding/decryption
logic used by this library in the mobile code as well, but that should
be simple enough.
This completes a major redesign of the Zulip login and registration
pages, making them look much more slick and modern.
Major features include:
* Display of the realm name, description and icon on the login page
and registration pages in the subdomains case.
* Much slicker looking buttons and input fields.
* A new overall style for the exterior of these portico pages.
This new feature makes it possible to request a different set of
initial data from the event_types an API client is subscribing to.
Primarily useful for mobile apps, where bandwidth constraints might
mean one wants to subscribe to events for a broader set of data than
is initially fetched, and plan to fetch the current state in future
requests.
- Add aggregated info to real-time updated presence status.
- Update `presence events` test case with adding aggregated
information to presence event.
- Add test case for updating presence status for user which
send state from multiple clients.
Fixes#4282.
This fixes a confusing issue where a user might try resetting the
password for an email account that in part of a different Zulip
organization.
Is a useful early step towards making Zulip support reusing an email
in multiple realms.
Fixes: #4557.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server. Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.
- Add script wrapper to communicate postfix pipe with django web server
over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
based on updated script.
- Add tests.
Tweaked by tabbott to adjust the directory and set better defaults.
Fixes#2421.
Rename 'zulip_internal' decorator to 'require_server_admin', add
documentation for 'server_admin', explaining how to give permission
for ./activity page.
Fixes: #1463.
This is basically just using the new check_dict_only everywhere, with
a few exceptions:
* New self.check_events_dict automatically adds the id field to avoid
duplicating it ~80 times.
* Set log=False for many of the testing action functions to remove the
timestamp field from their returned event dictionaries, since it's
not needed and is the result of a deprecated log_event function.
Wasn't sure if the subscription_field list in do_test_subscribe_events
could contain optional arguments, so I left the call to check_dict on
along with a TODO.
Fixes: #1370.
In this commit we add a logout wrapper so as to enable developers
to just do self.logout instead of doing a post request at API
endpoint for logout. This is achieved by adding a wrapper function
for the Django's client.logout contained in TestCase. We add this
by extending ZulipTestCase to have a logout function.
This is an incomplete cleaned-up continuation of Lisa Neigut's push
notification bouncer work. It supports registration and
deregistration of individual push tokens with a central push
notification bouncer server.
It still is missing a few things before we can complete this effort:
* A registration form for server admins to configure their server for
this service, with tests.
* Code (and tests) for actually bouncing the notifications.
In cases where old unread messages in the home view might have been
leaked (either due to bugs or unusual muting interactions), it's
theoretically possible for the first unread message in the home view
to be far older than the pointer.
Since the Zulip mobile app is loading messages following the
use_first_unread logic, we need to plug this gap.
Probably a longer-term solution will involve changing how
update_message_flags works to automatically advance the pointer, but
this change should make it possible for the mobile apps to
consistently use the `use_first_unread` mechanism for fetching the
latest home view messages.
With tweaks to the tests by tabbott.
Fixeszulip/zulip-mobile#422.
The previous logic was that anyone with a link to a file could send it
to other users, but only the owner could make a file realm-public.
This had some confusing corner cases.
The new logic is much simpler:
* Only the file's owner/uploader can include a file in a message for
the first time.
* Anyone with access to read a file can share it with others by
including it in messages they send.
* Once a file has been sent to a public stream, any user in the realm
can access it.
In this commit we fix the occasionally breaking tests for
test_home.HomeTest.test_bad_narrow which were the result of
us patching global settings in test_upload to add some new emails
to CROSS_REALM_BOT_EMAILS and not rolling back.
textsearch based full text search doesn't match text in link tag but
PGroonga based full text search can match text in link tag.
Without this change, highlighting text in link tag generates broken
HTML.
This commit makes sure that GitHubAuthBackend will only authenticate
using its own authenticate method. This is done by adding a new
Python Social Auth strategy which instead of calling authenticate
method of Django, calls the authenticate of the backend directly.
The problem this commit solves is that while authenticating through
GitHub backend, we were ending up getting authenticated through
ZulipDummyBackend. This might happen because the default strategy used
by Python Social Auth calls the authenticate method of Django which
iterates over all the backends and tries the authenticate methods
which match with the function arguments. The new strategy this commit
adds calls the authenticate method of GitHub backend directly which
makes sense because we already know that we want to authenticate with
GithHub.
The actual problem of why we are ending up on ZulipDummyBackend is
still a mystery because the function arguments passed to its
authenticate method are different. It shouldn't be called.
Earlier, a stack was being used to go through the message and search
for links. Because of this, in some cases the images were added to
the preview in reverse. Using a queue will keep the image previews in
the same order as they appeared in the message.
Fixes#4453.
This fixes an issue with a nondeterministic number of database queries
being used in fetching bulk messages from the database. The source of
the problem was that we were fetching _all_ messages, not just the 600
that had been created by the test, and thus if the set of streams
present in messages in the test fixtures (which is random) changes,
the number of streams used (and thus number of queries) would change.
Show a user friendly message to the user if email is invalid.
Currently we show a generic message:
"Your username or password is incorrect."
The only backend which can accept a non-email username is LDAP.
So we check if it is enabled before showing the custom message.
Rishi Gupta
neiljp (Neil Pilgrim)
Harshit Bansal
Durga Akhil Mundroy
James Rowan
Vaida Plankyte
Steve Howell
Jack Zhang
Vishnu Ks
Sarah
Cory Lynch
Tim Abbott
Aditya Bansal
Umair Khan
Franziska von der Goltz
Pweaver (Paul Weaver)
vaibhav
Yago González
Greg Price
sinwar
Sampriti Panda
Abhijeet Kaur
kunall17
Aditya Ramesh
Tommy Ip
K.Kanakhin
Maxim Averin
Ron Shafii
Ethan Estrada
Reid Barton
Elliott Jin
Andrew Archer
Rick Chern
umkay
Andrew Archer
kb0rg
Lech Kaiel
Matt Long
Lukasz Prasol
andrewallen00
Rohitt Vashishtha
Eeshan Garg
fionabunny
Raghav Jajodia
Neeraj Wahi
Brock Whittaker
Michael
Jacob Hurwitz
Kouhei Sutou