Commit Graph

12641 Commits

Author SHA1 Message Date
Steve Howell 69694b77fb casper: Improve common.js.
These changes prepare us for the casperjs upgrade:

    Extract init_viewport().
    Have then_log_out() do more explicit waiting.
    Add turn_off_press_enter_to_send().
2016-10-07 13:30:11 -07:00
Tim Abbott a4b2a6b6d4 puppet: Fix missing dependency for supervisor enabling.
You can't run `systemctl enable` on a service before the package
containing the service is installed.
2016-10-06 21:44:10 -07:00
K.Kanakhin 06e70e1fbd run-dev-proxy: Fix twisted py3 compatibility.
- All necessary strings was converted to bytestring
 - Added twisted as py3 dependency
 - Change type annotation for method getchild of class Resource
 - Remove activating python2 env section from run-dev.py script

Fixes #1256
2016-10-06 20:21:25 -07:00
umkay 78477ea071 Reorder the columns in analytics tables inherited from BaseCount.
This is primarily implemented through altering the migration file in
order to move the columns, but also we try to make the defaults a
little better for future tables inherited from BaseCount.
2016-10-06 17:51:01 -07:00
Diptanshu8 d7253b144c generate-secrets: Refactor to make development/production explicit.
generate-secrets.py now requires --development for development environment
setup or --production for production environment setup (and one of these
options is mandatory).

This solves the problem that it was somewhat easy to accidentally run
generate-secrets.py without the `-d` option while doing manual development
environment setup.

Fixes: #1911.
2016-10-06 17:12:49 -07:00
umkay 01324f2afe Fix aggregation to analytics summary tables.
There are a number of different stats that need to be propagated from
UserCount and StreamCount to RealmCount, and from RealmCount to
InstallationCount. Stats with hour intervals also need to have their day
values propagated. This commit fixes a bug in the summary table aggregation
logic so that for a given interval on a CountStat object we pull the correct
counts for the interval as well as do the day aggregation if required. We Also
ensure that any aggregation then done from the realmcount
table to the installationcount table follows the same aggregation logic
for intervals.
2016-10-06 08:46:33 -07:00
Rishi Gupta ff4d059595 Message editing: add explanation when past message edit deadline.
Adds "Topic editing only" along with the tooltip explanation to lower right
of message editing box when it's past the message content editing deadline.
2016-10-05 22:42:56 -07:00
Brock Whittaker cc8b8c1614 Separate default streams list from admin_tab template.
This separates default streams list settings into its own template in
the templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker 7be19dd860 Separate streams list from admin_tab template.
This separates streams list settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker b196fb9798 Separate bot list from admin_tab template.
This separates bot list settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker a56379e4d7 Separate deactivated user list from admin_tab template.
This separates deactivated user list settings into its own template in
the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker ecb6c7df40 Separate user list from admin_tab template.
This separates user list settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker d2f3e9eb7f Separate emoji settings from admin_tab template.
This separates emoji settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker 403c6322ae Separate organization settings from admin_tab template.
This separates organization settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker b81967d6ef Separate account settings from settings_tab template.
This separates account settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker 26fda72157 Separate UI settings from settings_tab template.
This separates UI settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker 330edddeff Separate alert word settings from settings_tab template.
This separates alert word settings into its own template in the
templates/settings folder.
2016-10-05 22:27:15 -07:00
Brock Whittaker 68b8a7b40d Separate bot settings from settings_tab template.
This separates bot settings into its own template in the
templates/settings folder.
2016-10-05 22:27:14 -07:00
Brock Whittaker aca78df56f Separate notification settings from settings_tab template.
This separates notification settings into its own template in the
templates/settings folder.
2016-10-05 22:27:14 -07:00
Brock Whittaker ff3d5ca4db Separate display settings from settings_tab.handlebars.
This separates the display settings module from the
settings_table.handlebars template.

Additionally, it fixes the node tests to search in the
static/templates/settings directory and initialize any templates in there
while running tests on the settings templates.
2016-10-05 22:26:40 -07:00
Brock Whittaker 3efe3601b8 Allow for templates/settings to be read and compiled.
This allows for the handlebars templates in the template/settings
subdirectory to be found and compiled.
2016-10-05 22:24:22 -07:00
Tim Abbott 82764d0215 run-dev: Display the port 9991 warning in colored text. 2016-10-05 22:07:46 -07:00
Feorlen e09c27f1ae run-dev.py: Log what service is being started on which port.
The new messages make it more obvious which services are started
from run-dev.py, and explicitly call out where to access the web
proxy to reach the Zulip web UI. This is a common confusion for
new administrators/developers. Messages are output before the
processes are launched, as run-dev.py does not currently have a
way to know if they started successfully.

Example output:

Starting Zulip services on ports:  web proxy: 9991, Django: 9992, Tornado: 9993, webpack: 9994
Note: only port 9991 is exposed to the host in a Vagrant environment.

Alternate behavior for automated testing:

If run-dev.py is invoked with --test, don't include the webpack
port as it isn't used.

Tested on Ubuntu 14.04, by running run-dev.py at a shell prompt and
via the test-all script.

Fixes #1861
2016-10-05 22:07:46 -07:00
Tim Abbott d0f295c77f docs: Clarify commit message portion of version control docs. 2016-10-05 19:28:21 -07:00
Rishi Gupta d387012bc6 Fix EXTERNAL_HOST computations in test_settings.py.
test_settings.py was setting EXTERNAL_HOST after importing settings.py,
which has several variables (like SERVER_URI) that are computed from
EXTERNAL_HOST.

[tweaked by tabbott to add comments explaining the story here].
2016-10-05 19:04:55 -07:00
Tim Abbott 5357b286b2 docs: Fix typo in spelling of generate_secrets. 2016-10-05 17:47:14 -07:00
Tim Abbott 273c17a072 update_analytics_counts: Add missing future imports. 2016-10-05 17:13:46 -07:00
umkay 5d0bed8673 Add script to clear analytics tables. 2016-10-05 17:11:13 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
Tim Abbott 498317d533 Fix renaming streams on web to use new API.
In 142dce2cd4, we replaced the API for
renaming a stream, but failed to actually update the javascript to use
the new API.
2016-10-04 21:48:38 -07:00
Tim Abbott 3973ae5dbb update_analytics_counts: Fix buggy argument parsing. 2016-10-04 20:43:19 -07:00
Tim Abbott a2b91221d0 upgrade-zulip: Only remove zproject/local_settings.py if symlink.
This fixes a problem where if we're deploying from git,
local_settings.py ends up part of the diff in the deployed git
repository.
2016-10-04 20:41:17 -07:00
Tomasz Kolek 7df50cee34 docs: Fix paths in webhooks part of integration-guide.
We forgot to update the documentation when we rearranged the webhooks
tests to be split across several files.

[mostly rewritten by tabbott]
2016-10-04 20:09:58 -07:00
Tim Abbott 49bea381b7 zjsunit: Fix running stream_data and node tests individually.
These tests failed if you ran them along with:
 `ReferenceError: $ is not defined` errors.

Fixes #1909.
2016-10-04 18:38:17 -07:00
umkay b4108f7a5f Rerun add-apt-repository for ppa:groonga/ppa
On occasion, provisioning will fail because groonga is not added. Add a
check to see if the command fails and retry.
2016-10-04 18:21:36 -07:00
Tim Abbott 9b30594bb6 run-casper: Fix buggy comments about subdomains testing. 2016-10-04 18:16:53 -07:00
umkay d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Steve Howell 43e21f6fb4 Remove unused bugdown global from models.py.
Fixes: #1379

(It was really a series of fixes leading up to this.)
2016-10-04 11:35:03 -07:00
Steve Howell b2f84f0fa4 Move render_markdown() to lib/message.py.
This removes a bugdown circular dependency.
2016-10-04 11:34:53 -07:00
Steve Howell b81a961fdd Remove Message.set_rendered_content(). 2016-10-04 11:31:21 -07:00
Steve Howell d21d443d29 Deprecate render_old_messages command. 2016-10-04 11:31:21 -07:00
Steve Howell 7fb992dba3 Simplify and "fix" render_old_messages management command.
The command to render old messages now looks for all messages
not matching the bugdown version, and it no longer directly calls
into model code.  We should still be extremely cautious about
using this code.
2016-10-04 11:31:20 -07:00
Steve Howell 6b71f5bd5f Inline most calls to set_rendered_content().
This is part of breaking the circular dependency on
bugdown in models.py.  A subsequent commit will fully
kill off set_rendered_content().
2016-10-04 11:31:20 -07:00
Steve Howell 583a6bbadd Extract zerver/lib/message.py.
This pulls message-related code from models.py into a new
module called message.py, and it starts to break some bugdown
dependencies.  All the methods here are basically related to
serializing Message objects as dictionaries for caches and
events.

    extract_message_dict
    stringify_message_dict
    message_to_dict
    message_to_dict_json
    MessageDict.to_dict_uncached
    MessageDict.to_dict_uncached_helper
    MessageDict.build_dict_from_raw_db_row
    MessageDict.build_message_dict

This fix also removes a circular dependency related
to get_avatar_url.

Also, there was kind of a latent bug in Message.need_to_render_content
where it was depending on other calls to Message to import bugdown
and set it globally in the namespace.  We really need to just
eliminate the function, since it's so small and used by code that
may be doing very sketchy things, but for now I just fix it.  (The
bug would possibly be exposed by moving build_message_dict out to the
library.)
2016-10-04 11:31:20 -07:00
Steve Howell ac994fdd51 Move three functions from models.py to lib/cache.py.
I move these three functions to lib/cache.py:

    to_dict_cache_key_id
    to_dict_cache_key
    flush_message

This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
2016-10-04 11:31:20 -07:00
Tim Abbott 279f805448 run-dev: Automate passing --interface='' for vagrant environment.
Fixes #877.
2016-10-04 10:48:26 -07:00
Steve Howell 69fa60080c tests: Test get_prereg_user_by_email(). 2016-10-04 10:35:43 -07:00
Steve Howell ad149141f3 tests: Test Subscription.__unicode__(). 2016-10-04 10:35:43 -07:00
Steve Howell 5ae8cff289 code cleanup: Remove unused Attachment methods.
This removes some unused code on Attachment.  Some of these
methods might be useful in a manage.py shell, but without tests,
they are not very trustworthy, and the Attachment model isn't that
complicated to write raw Django queries against.
2016-10-04 10:35:43 -07:00