Commit Graph

3885 Commits

Author SHA1 Message Date
Steve Howell fa31ad35c9 Fix display of changed avatars in old messages (page_params).
Our client code will now receive avatar_url in
page_params.people_list during page load, so it will be
able to use more current urls for old messages (the client
already had some logic for that and was just missing the
data).

We also add avatar_url to the realm_user/add event.

When we change the avatar, we make sure to always send a
realm_user/update event (even for bots).

We also needed to add avatar_version and
avatar_source to our active users cache.
2017-02-22 07:57:03 -08:00
Steve Howell 9d5a631650 Add test_change_avatar_fields().
This adds a temporary, insignificant change to apply_events().
2017-02-22 07:57:03 -08:00
Tim Abbott 160891381a test_events: Add comments for do_test_subscribe_events. 2017-02-22 00:34:12 -08:00
Tim Abbott a580e7088d email: Send an event when email addresses change. 2017-02-21 23:42:46 -08:00
Ayush Jain 455c1919fc Add customizable invite-new-user text.
This makes life a lot easier for people inviting users to a new Zulip
organization, since they can give some form of context now.

Modified by tabbott to clean up CSS, backend code flow, and improve
the formatting of the emails.

Fixes: #1409.
2017-02-21 22:35:01 -08:00
Tim Abbott cf96b1b873 generate_realm_creation_link: Clean up instructions. 2017-02-21 20:19:16 -08:00
Tim Abbott 353f969992 test_events: Add missing print_function py3 import. 2017-02-21 14:26:31 -08:00
Steve Howell 192805903d Make events test more robust for apply_events calls.
We now make tests that call EventsRegisterTest.do_test()
explicitly specify whether calls to apply_events() would
change the state of initially fetched data.  Generally
these tests exist to test that logic (as well as verifying
schemas of events), so if they stop testing that logic, it
is usually a broken test.

Some tests are exempted from the check here, because I think
they don't really change state--such as updating messages or
notifications.  You can set state_change_expected to False
for those tests.

For all the tests that deal with flipping boolean flags, I
set their value to False before calling do_test twice now.

For the authentication backends, I mock the settings so that
more backends are "supported" and therefore part of the event
and the fetched state.

Finally, for the bot tests, I make sure to use a bot the user
can access.
2017-02-21 11:43:09 -08:00
Tim Abbott 20f9c04ab5 decorator: Add logging data to zulip_login_required.
This fixes an issue that many logged=in pages such as /stats did not
correctly report either the connecting client or the user in server
logs.
2017-02-20 21:15:44 -08:00
Tim Abbott f52d812a71 events: Extract apply_event helper.
This mostly just saves us a level of messy indentation.
2017-02-20 11:16:35 -08:00
Tim Abbott e4fbad95c6 events: Clean up confusing comment about subscribers. 2017-02-20 11:07:09 -08:00
Tim Abbott dc0f6413f3 events: Fix include_subscribers=False race conditions.
The original include_subscribers implementation did not correctly
update the apply_events code path to avoid adding 'subscribers' dicts
to things.  This corrects that oversight.
2017-02-20 11:07:09 -08:00
Tim Abbott 7a930afa07 events_register: Don't include subscribers in API data by default.
There's a new option, `include_subscribers`, that controls whether the
API sends down subscriber data for the various streams you are
subscribed to.

This has significant performance savings for large realms with naive
clients, and saves a bunch of bandwidth as well.
2017-02-20 11:07:09 -08:00
Tim Abbott 1cbc86499e events_register: Eliminate unnecessary api_events_register wrapper. 2017-02-20 11:07:09 -08:00
Tim Abbott b231bf70eb uploads: Remove display_recipient from fetched data.
This fixes a performance regression loading the Zulip homepage.

While it decreases the utility of the display of messages, it's only
so much loss (because the display recipient for PMs was totally broken
anyway).
2017-02-19 22:19:39 -08:00
Umair Khan dbe609d515 testing: Copy code from unittest.TestSuite.run.
This allows us to use setUpClass.
2017-02-19 21:04:06 -08:00
Umair Khan 128beb910b testing: Use TestResult in run_test.
Internally, Django uses TestResult to gather results from testcases.
This commit upgrades the run_test function to use TestResult to compile
result.
2017-02-19 21:04:06 -08:00
Steve Howell 96fc6f2372 Remove deactivated streams from the DefaultStream model.
If a stream is deactivated, we now remove it from the
DefaultStream model on the back end.
2017-02-19 18:06:20 -08:00
Kartik Maji 1a697b6e02 Add frontend to show message edit history.
Fixes #268.

Modified significantly by tabbott to:
* improve code cleanliness / repetition
* add missing translation tags
* move code into message_edit.js
* correspond with the new backend.
* not display the option for messages only topic-edited
2017-02-19 17:41:06 -08:00
Tim Abbott 98894eb759 edit_history: Reverse the order of edit history list. 2017-02-19 16:47:07 -08:00
Tim Abbott eadd6fb3c6 message_edit: Include an initial entry in view output.
This makes it super easy for frontend code using this view code to
produce a nice display of the history.

This also fixes an off-by-one error with the timestamps.
2017-02-19 16:46:47 -08:00
Tim Abbott 8da7976058 queues: Add new system for managing rabbitmq per-queue work.
Our lists of rabbitmq queues was likely to end up out of date, since
there was nothing enforcing that the various lists of queues were
correct or the same as each other.
2017-02-19 16:18:37 -08:00
Tim Abbott 515340ed00 Add message edit history backend with tests.
Based on work by Kartik Maji in #1204.

This has a few significant changes from the original version:
* We correctly handle filling in data for topic edits
* Has a complete test suite verifying correctness of the logic
* Currently, it doesn't include a special "start" entry

Things we may want to further change include:
* Adding a special "start" entry.
* Reversing the order of the history data returned for clarity.
2017-02-19 16:13:35 -08:00
Tim Abbott 498f6782a2 EditMessageEdit: Add complete test of edit history. 2017-02-19 16:13:35 -08:00
Tim Abbott 976868cf01 message_edit: Store which user edited a given message.
This is important for, in the future, being able to display who edited
the topic of a message if that wasn't the person who originally sent
the message.
2017-02-19 16:13:04 -08:00
Tim Abbott f1d82af191 Include prev_rendered_content_version in edit events. 2017-02-19 15:23:08 -08:00
Tim Abbott 6ad883fb61 docs: Update queuing documentation for new templates. 2017-02-19 13:12:34 -08:00
Tim Abbott 0383ff1dfd lint: Fix homeassistant PEP8 violations. 2017-02-18 21:51:14 -08:00
ausDensk 14024963f2 Add a simple Home Assistant integration for Zulip. 2017-02-18 21:18:10 -08:00
Tim Abbott 32bfebeb7a mypy: Fix inconsistencies in use of *args/**kwargs. 2017-02-18 18:39:44 -08:00
Tim Abbott 473c0ee1fe mypy: Remove now-unused type: ignores. 2017-02-18 17:01:01 -08:00
Tim Abbott b63a8a7880 integrations: Fix mypy error. 2017-02-18 15:29:43 -08:00
Tim Abbott b30fb37037 Fix URL routing for users with email addresses starting with "me".
Our URL routing previously attempting to segment the /users/ endpoint
namespace into /me (affecting yourself) or /username@domain (affecting
other users) by regular expressions incorrectly, specifically in the
case of email addresses starting with `me`.  This prevented various
admin actions like removing a user as an organization administrator.
2017-02-18 15:26:04 -08:00
Tim Abbott bbecd41376 help: Fix link to index page being present on index page.
Fixes #3613.
2017-02-18 14:59:43 -08:00
Tim Abbott 6ba1cd797b Fix regression in whitespace stripping at the start of messages.
This fixes a regression in 4060a97656,
which incorrectly stripped whitespace at the start of messages as
well.

Fixes #3719.
2017-02-18 14:47:28 -08:00
Steve Howell b9ec2545bb Simplify and speed up stream deactivation.
This is a fairly risky, invasive change that speeds up
stream deactivation by no longer sending subscription/remove
events for individual subscribers to all of the clients who
care about a stream.  Instead, we let the client handle the
stream deactivation on a coarser level.

The back end changes here are pretty straightforward.

On the front end we handle stream deactivations by removing the
stream (as needed) from the streams sidebar and/or the stream
settings page.  We also remove the stream from the internal data
structures.

There may be some edge cases where live updates don't handle
everything, such as if you are about to compose a message to a
stream that has been deactivated.  These should be rare, as admins
generally deactivate streams that have been dormant, and they
should be recoverable either by getting proper error handling when
you try to send to the stream or via reload.
2017-02-17 15:53:22 -08:00
Steve Howell b215a23456 bug fix: Fix stream deactivation being super slow.
This fix prevents stream deactivation from being basically
un-usable for medium to large sites.  Instead of calling
bulk_remove_subscriptions one at a time for every individual
member of the realm, we call it once for all the users that
care about the stream.  This change makes a huge difference, but
the feature is still a bit clunky, and we should only temporarily
revert to this fix if future, more-invasive fixes have flaws.

Fixes #3631.
2017-02-17 15:48:10 -08:00
Tim Abbott c61d0a78f4 home: Remove unused get_client import. 2017-02-17 15:35:38 -08:00
Tim Abbott 7a5065da62 views: Stop hardcoding the 'website' client for the / endpoint.
We were apparently incorrectly harcdoding the client for the main
logged-in site loading to website, rather than using the existing
logic that could sort out the desktop apps.
2017-02-17 15:20:42 -08:00
Tim Abbott 8487a24a6c socket: Hardcode website message sender for fake messages. 2017-02-17 15:20:32 -08:00
Tim Abbott 9072d3c350 socket: Transmit HTTP_USER_AGENT for websockets sending.
This significantly simplify the logic for our logging process, making
it the case that websockets message sending requests always are logged
as having the exact same client as a normal AJAX request from that
server.
2017-02-17 15:19:21 -08:00
Tim Abbott c9126e772e decorator: Don't block ZulipElectron name in json views.
This fixes the logging of the ZulipElectron client in server logs.
Message sending is still logged as "website"; that will be fixed soon.
2017-02-17 15:19:10 -08:00
Tim Abbott 5c911c44d6 MessageSenderWorker: Reformat environment setup. 2017-02-17 15:00:57 -08:00
Umair Khan a9bc625dda testing: Serialize test_upload.
This commit makes test_upload tests compatible with parallel model.
2017-02-17 12:40:39 -08:00
Umair Khan b7d22e3308 testing: Isolate test_patch_bot_avatar.
This commit changes test_patch_bot_avatar to upload avatars to a
different directory so that there is no race condition when tests are
run in parallel mode.
2017-02-17 12:18:43 -08:00
Harshit Bansal 21dccaedd1 test_events.py: Add a test for the `do_change_realm_alias()` function. 2017-02-17 11:55:15 -08:00
Harshit Bansal fec145fc0c zerver/models.py: Add comments in 'get_realm_by_email_domain()` function.
Add necessary comments explaining our query sequence.
2017-02-17 11:55:15 -08:00
Steve Howell ad24133b94 Have functions in lib/avatar.py use avatar versions.
In some cases here we simplify things by calling avatar_url()
instead of get_avatar_url(), when we have a user_profile record
handy.  For other cases we pass in an extra avatar_version
parameter to get_avatar_url(), including from avatar_url().
2017-02-17 10:19:56 -08:00
Steve Howell 3a04831793 Add avatar_version to active_bot_dict_fields. 2017-02-17 10:19:56 -08:00
Steve Howell 65a4eb8ec8 Add sender_avatar_version to message caches.
We will use this in computing avatar URLs.
2017-02-17 10:19:56 -08:00
Adarsh S 3c2c0c67b9 Bump user_profile.avatar_version when we change avatars.
We have a field called user_profile.avatar_version that will
track avatar versions and be used tactically in avatar urls
to get browsers to refresh their caches (in future commits).

This commit bumps the avatar version when we update avatars.

We do this in do_change_avatar_fields(), which was
do_change_avatar_source() before this change.

Adarsh did the initial work here, and Steve Howell (showell) also
made changes.
2017-02-17 10:19:56 -08:00
paxapy 9a5179c460 Add support for managing and deleting attachments.
Modified substantially by tabbott to fix tons of issues.

Fixes #454.
2017-02-16 23:44:44 -08:00
Tim Abbott f528af2be0 Cleanup some unnecessary calls of get_active_user_dicts_in_realm. 2017-02-16 23:29:07 -08:00
Tim Abbott 51313b6b29 storage: Fix minifying CSS/JS files with unicode in them.
Previously, this header construction work would fail on Python 2 in
the case that there were unicode characters in the input.
2017-02-16 22:24:14 -08:00
Tim Abbott d564a76f8e alert_words: Consistently clean whitespace for alert words.
This fixes some gaps in handling of whitespace in alert words.
2017-02-16 21:06:18 -08:00
Raghav Jajodia c1dfa348a1 alert_words: Trim whitespace around alert words.
"Add a new alert word" box now displays an alert when filled with only spaces.
Fixes #3369
2017-02-16 21:06:18 -08:00
Tim Abbott 99c0b0e79f Fix favicon notifications issues for messages sent by oneself.
In Zulip, we mark messages that you send to yourself as read if and
only if they were sent from a known client that represents a human
user use case.  The purpose of this logic is to (1) mark messages
humans send as read while (2) still making it convenient to have a bot
that sends messages to yourself for something like Google calendar,
where you actually want to read those messages.

It's possible that we want to move the control for this behavior into
a client-specific flag rather than doing this off User-Agent.

Fixes #3694.
2017-02-16 13:21:43 -08:00
Steve Howell 5f2e320fee Fix test_valid_api_key_if_user_is_on_wrong_subdomain.
This test would fail if settings.RUNNING_INSIDE_TORNADO
was True, which seemed to happen due to other tests changing
that setting, although I did not fully investigate.
2017-02-15 14:57:01 -08:00
Steve Howell d406d34fe0 Use user_id in admin_user_list.handlebars.
For our user administration, we now primarily work with user ids
that get put into data-user-id attributes.  We still put emails in the
tags to make our Casper tests easy to maintain.

This requires a minor change to the back end to pass down user ids
for the /users endpoint (in get_members_backend).
2017-02-14 23:07:44 -08:00
Umair Khan 6db4879f9c testing: Clear cache in queries_captured.
This results in a more deterministic result when we count queries.
2017-02-13 14:24:48 -08:00
Tim Abbott df7b10b7f2 test_runner: Hackishly fix broken failed status.
Something in c14e981e00 broken test
failures being reported properly; this isn't the right fix but works
and will let us avoid reverting the original change until it can be
fixed properly.
2017-02-11 23:02:47 -08:00
Tim Abbott 4060a97656 messages: Strip trailing whitespace in message contents.
I dug into why we never did this before, and it turns out we did, but
using `$.trim()` (which removes leading whitespace as well!).  When
removing the `$.trim()` usage.

Fixes #3294.
2017-02-11 23:01:22 -08:00
Robert Hönig 6ee845d027 Add html versions of the invite and signup mails
This commit adds html versions of the invite and signup mails and renames
the existing .txt files to the preferred file extensions '.subject', '.html'
and '.txt'. The html versions of the mails are being sent along with the
text-only versions by the 'send_confirmation' function.
This fixes #3134.
2017-02-11 17:08:57 -08:00
Tim Abbott 680d30adc2 docs: Add comments linking to new events doc. 2017-02-11 16:59:28 -08:00
Tommy Ip c7e33c6c9f optimization: Use Python to test management commands.
The original test was written in shell script which launches a new
django instance for every tests. By doing it in Python, we avoid
the overhead and reduce the test time to <1 second.

Fixes #3620.
2017-02-11 13:48:16 -08:00
Tim Abbott 46226bad21 presence: Fix mypy errors in new feature. 2017-02-10 23:57:28 -08:00
Tim Abbott eeca69cb4b mypy: Clean up more optional types. 2017-02-10 23:53:44 -08:00
Tim Abbott aee81b702c add_subscriptions_backend: Cleanup type of principals. 2017-02-10 23:53:44 -08:00
Tim Abbott c9a782ff75 Simplify zerver/views/users.py:get_stream_name. 2017-02-10 23:53:44 -08:00
Tim Abbott 2bba7755e3 decorator: Improve user agent parsing. 2017-02-10 23:53:44 -08:00
Tim Abbott bc63407e6a decorator: Cleanup api_key_legacy code readability. 2017-02-10 23:53:44 -08:00
Tim Abbott b81fd407e8 mypy: Fix several Optional typing errors. 2017-02-10 23:53:44 -08:00
Tim Abbott 6d00d4d2b1 mypy: Clean bugdown use of Optional. 2017-02-10 23:53:44 -08:00
Tim Abbott e746868375 mypy: Fix optional typing usage in rendering code path. 2017-02-10 23:53:44 -08:00
Tim Abbott 4e20b622ad bugdown: Remove unused height argument to add_a. 2017-02-10 23:53:44 -08:00
Tim Abbott 9f0c1db430 mypy: clean up strict optional errors in upload.py. 2017-02-10 23:53:44 -08:00
Tim Abbott 2140d2963d UserPresence: Cleanup confusing to_presence_dict arguments. 2017-02-10 23:53:44 -08:00
Tim Abbott 1c0f65cbec mypy: Fix some incorrect optional tags. 2017-02-10 23:53:44 -08:00
Tim Abbott bb5d81281c mypy: Upgrade to new package name and version 0.571.
Fixes #3448.
2017-02-10 23:53:44 -08:00
Tim Abbott 650469ead6 mypy: Make zerver/lib/ccache.py support python 3. 2017-02-10 23:53:44 -08:00
Tim Abbott 923f4ddd80 models: Remove nullable property from RealmAlias.realm. 2017-02-10 23:53:44 -08:00
Tim Abbott 71af0f7e2e Add endpoint to fetch presence data for a single user.
This is an experimental API subject to its data format being changed.

Fixes #3638.
2017-02-10 23:52:56 -08:00
Tim Abbott ac8b661a8f presence: Extract get_status_dicts_for_query. 2017-02-10 23:52:56 -08:00
saisrivathsa b867ac3496 Extract zerver/lib/events.py from actions.py with event registration.
This moves do_events_register, fetch_initial_state_data and friends to
a new file.

Modified significantly by tabbott for correctness and to remove unused
imports.

Fixes #3635.
2017-02-10 16:50:43 -08:00
Umair Khan c14e981e00 testing: Conform runner to Django architecture.
In Django, TestSuite can contain objects of type TestSuite as well
along with TestCases. This is why the run method of TestSuite is
responsible for iterating over items of TestSuite.

This also gives us access to result object which is used by unittest
to gather information about testcases.
2017-02-10 16:01:43 -08:00
Umair Khan e5a16ceb0a testing: Use failfast instead of fatal_errors.
`failfast` has the same meaning as `fatal_errors` in Django's test
runner.
2017-02-10 16:01:43 -08:00
Umair Khan 7743f74180 Do not append to INSTRMENTED_DATA directly.
Use append_instrumentation_data to append data to the INSTRUMENTED_DATA.
This gives us a layer of abstraction when we need to add instrumentation
data from other modules e.g. while running tests in parallel mode.
2017-02-10 16:01:43 -08:00
Umair Khan 78768a2ba9 Add process_instrumentation_data function.
This function can be used to perform processing on instrumentation data.
For example, this can be used to send the instrumentation data gathered
in the test suite running in the child process to the parent process for
aggregation.
2017-02-10 16:01:43 -08:00
Umair Khan 00f8239563 Clean code of send_test_email command.
* Derive from sendtestemail command of Django.
* Remove unwanted imported.
* Allow test email to admin and managers.
2017-02-10 16:01:43 -08:00
Umair Khan ef0d2a4bb5 logging: Use django.server to filter 200 and 304.
Previously, we were monkey patching the runserver command
in zerver/management/commands/rundjango.py for this.
2017-02-10 15:55:17 -08:00
Brock Whittaker 1143ed7219 redesign: Change /#settings and /#administration to an overlay.
This also adds a box-shadow to the #deactivate_self_modal so that it
looks similar to the old backdrop.
2017-02-09 23:35:10 -08:00
Harshit Bansal 8ae54ddb99 actions.py: `restricted_to_domain` should be False if there are no aliases.
Having `restricted_to_domain` set to True if there are no more aliases
left means the user is either confused or forgot to set it to False. It
should be set to False automatically when the last alias is deleted.
2017-02-08 22:14:43 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Rishi Gupta a16c48f0c4 actions.py: Change do_remove_realm_alias to take a RealmAlias.
Ensures that this function doesn't throw an error / prevents putting in an
incorrect realm or domain argument.
2017-02-08 21:15:28 -08:00
Tim Abbott 0c363dffca lint: Ban use of deprecated assertEquals. 2017-02-08 16:38:43 -08:00
Umair Khan c585fa6eb4 change-email: Delete display recipient cache. 2017-02-07 21:49:31 -08:00
Tim Abbott 84b18f865a users: Verify full names explicitly in account registration.
I believe this completes the project of ensuring that our recent work
on limiting what characters can appears in users' full names covers
the entire codebase.
2017-02-07 20:20:32 -08:00
Tim Abbott 56cecc4891 users: Verify full names explicitly in user creation.
This fixes an issue where users could be created with an invalid name
(introduced only a couple commits ago when we added character set
restrictions).
2017-02-07 19:54:30 -08:00
Tim Abbott 2283b5fc91 users: Consolidate name change enforcement logic.
This has the side effect of fixing an issue where one could edit a
bot to have an invalid name.
2017-02-07 19:45:21 -08:00
Ritwik Srinivas 74b68f6bbc Adds banned characters in name function
Disallows you from putting the characters @, *, `, and > and " in
your name. Added test cases similar to the MAX_NAME_LENGTH check

Copied initial code from:
https://github.com/zulip/zulip/pull/2473
2017-02-07 19:31:14 -08:00