This keeps the examples in line with our actual codebase.
Also while I'm here revise it to explain the actual motivation for our
use of `env`, and to correct some subtle details -- it's actually the
kernel that interprets the shebang (as visible in e.g. a `strace` log),
not the shell, and when the program is executed as `./my_program.py`
the exact name including `./` is passed to the interpreter.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2. In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.
One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2. See discussion on the respective previous
commits that made those explicit. There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
This works fine in my testing -- I followed it on a fresh `trusty` VM,
after just getting SSL certs with our LetsEncrypt instructions, and
the install completed successfully.
And in the source tree, the only evidence I can find of a potential
remaining dependency on the `/root/zulip` path is the Nagios config in
`puppet/zulip_ops/`. That's actually already broken, in that it
depends on `/root/zulip/api/`, so we'll have to sort that out;
and in any case, it doesn't matter to most people installing Zulip.
Add a mention that the install script will move the just-unpacked
directory out from under you. While we're here, add a few words about
where the deployed code is laid out.
Text of those last words tweaked by tabbott.
This follows up on 207cf6302 from last year to clean up cases that
have apparently popped up since then. Invoking the scripts directly
makes a cleaner command line in any case, and moreover is essential
to how we control running a Zulip install as either Python 2 or 3
(soon, how we always ensure it runs as Python 3.)
One exception: we're currently forcing `provision` in dev to run
Python 3, while still running both Python 2 and Python 3 jobs in CI.
We use a non-shebang invocation to do the forcing of Python 3.
Since we're auto-detecting the value anyway, there's no reason it
can't be moved to DEFAULT_SETTINGS.
This lets us remove some clutter from the installation documentation.
This field is convenient for bankruptcy checks. Clients could
calculate it from page_params.unread_msgs before this change, but
it would kind of a painful calculation.
To add count, we had to simplify the mypy annotations, which weren't
really accurate before.
Update Email, Beanstalk, Hubot, JIRA, and Trello integrations
links.
The Hubot integrations section (/integrations#hubot-integrations)
was removed in an earlier redesign of /integrations. This commit
replaces the link with the hubot-scripts organization on
Github, which displays the comprehensive list of all integrations
available via Hubot.
Fixes#5875.
In some of these contexts, we may still be *using* the Python 2
version, but at least this should eliminate running into
`ImportError`s one by one in scripts that run outside a virtualenv,
as we update their shebangs to refer to Python 3.
Several Python libraries we use don't come in Python 3 versions on
trusty: gevent, boto, twisted, django, django-tagging, whisper.
The latter two don't come in Python 3 versions even on xenial.
So some work required before we can actually switch the code that
relies on those libraries to run as Python 3 -- probably the best
solution will be to backport them all in our apt repo. (All but
`whisper` are packaged in zesty; `whisper` upstream just grew Python 3
support this year.)
We are phasing out the following in tests:
add_dependencies - this is just kind of a clunky UI
require - normal JS requires cause test leaks
In order to plug require leaks, we are effectively doing what
we always have done inside of add_dependencies, which is to
keep track of which modules we have done `require` on, and
these get cleared between tests.
Now we just use `zrequire` every time we want to pull in real
code to our global namespace.
I think soon we'll put the Python 3 venv at `/srv/zulip-venv` and
make `/srv/zulip-py3-venv` just an alias to that (then remove the
alias too), but for now the old name is helpful for spotting places
that need an update.
The `/srv/zulip-venv` name still appears in codepaths used by Travis
tests.
From the line in tools/provision it should trickle to the rest of the
scripts. This works since almost all the python scripts have been linted
to be generic.
Proof that the setup is python3 only: with this commit, within the
vagrant container env, /srv/zulip-venv is no longer present and
`./tools/run-dev.py` runs just fine.
[gnprice: Added `rm -f` and warning message, and made small edits.]
This makes supervisor see the service as cheerfully running
and let it alone, rather than constantly retry starting it.
Because the crash/restart loop means repeatedly spending a
couple of seconds loading Django and the app, separated by
brief periods while supervisor notices the crash and acts
on it, it was actually consuming about 30-50% CPU on the
zulipchat.com staging server.
This pulls together what's covered in detail in several
longer pages, and gives us a page that can serve as a good
drop-in replacement for https://zulip.org/server.html .
Also tweak a couple of related bits for clarity and orthography.
We are adding a new list of unread message ids grouped by
conversation to the queue registration result. This will allow
clients to show accurate unread badges without needing to load an
unbound number of historic messages.
Jason started this commit, and then Steve Howell finished it.
We only identify conversations using stream_id/user_id info;
we may need a subsequent version that includes things like
stream names and user emails/names for API clients that don't
have data structures to map ids -> attributes.
After editing this file for the previous commit, I couldn't unsee
"low-traffic the". So I fixed that, saw more issues, and then just
went through the page for everything that was similarly trivial to fix.
This allow the webbpack dev server to properly reload JavaScript modules
while running in dev without restarting the server. We need to connect
to webpack-dev-server directly because SockJS doesn't support more than
one connection on the same host/port.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization. It controls access through 2
key mechanisms:
* For long-ago edited messages, get_messages removes the edit history
content from messages it sends to clients.
* For newly edited messages, clients are responsible for checking the
setting and not saving the edit history data. Since the webapp was
the only client displaying it before this change, this just required
some changes in message_events.js.
Significantly modified by tabbott to fix some logic bugs and add a
test.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
`emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
package.
Bumps PROVISION_VERSION.
Fixes: #4730.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
In this commit we basically start to override the request method of
httplib2.Http() to raise an exception whenever it is called i.e.
a trial is made to access the network from test suits.
Fixes: #1472.
This old third party library added support
for a "mousewheel" event to detect scrolling.
However, it is not compatible with jQuery 3
and is obsolete now that there is a standard
"wheel" event that accomplishes the same thing.
I found the new paragraph from 74d83cc47 somewhat hard to follow,
so here's another version. Also try to make the structure of
the rest of the section around it somewhat more clear.
This commit adds a new linter which runs from tools/travis/backend.
It runs over the translations.json file and checks if any of the
translatable string contains handlebars in it.
Fixes#5544
This started as a PSA in the form of a series of chat messages in
`#general` on chat.zulip.org; putting them here, with some editing,
to make their value more durable.
Also rearrange this doc slightly so that it's not specific to
the server codebase, except in a few explicit spots.
The bit that's for authors should probably be somewhere else.
I think there isn't right now a great natural spot for it --
probably the top of docs/git-guide, some parts of docs/version-control,
and that paragraph here should all turn into a top-level "guide
to submitting code to Zulip" doc, which would link to the rest
of docs/git-guide and to some other resources. Leaving that
for another day.
This follows up the recent commit
3d1d09b3d docs: Remove discussion of old Django templating engine.
with a small grammar fix and removing another vestige of making the
distinction between Jinja2 and Django templates, and also rearranges
the logic slightly to reflect that backend and frontend templates
have separate sections.
Probably a bigger restructuring is in order to help the reader
navigate through all the good content in this doc, but that's
a bigger job for another day.
- Remove `perfect-scrollbar` from `static/third` and fetch it from npm.
- Upgrade `perfect-scrollbar` to 0.7.1.
- Bump up the `PROVISION_VERSION` to 5.6.
Changed `wheelSpeed` in "static/js/scroll_bar.js" to 0.5, because when it
20, the scrollbar scrolls very fast.
Changed 'wheelSpeed' in "static/js/emoji_picker.js" from 25 to 0.68
(based on tabbott's testing of scrolling through the emoji list).
Part of #1709.
Introduce Swagger UI and the Swagger/OpenAPI specification. Explain
the structure of zulip.yaml and show examples of different sections
of the file.
This is a new file in /docs not yet included in the Read the Docs
table of contents. Where it should go should be determined as we iterate
on the Swagger UI integration and expand REST API doc coverage using it.
For more on Swagger UI and how Zulip uses it, see:
https://github.com/zulip/zulip/issues/3474https://github.com/zulip/zulip/pull/3397
With some minor tweaks to advertise this by tabbott.
The formatting was coming out pretty badly on readthedocs,
and also I think the exposition could be a little clearer.
I still don't love how the formatting comes out, but
I think this is pretty OK.
There were 2 things wrong here:
(1) The new emoji cache directories weren't being created properly
(2) We weren't downloading the new emoji sprite sheets.
I think based on this experience, we should definitely invest in
moving more platforms to use provision.py.
Fixes#5160.
I faced this problem many a times, might be of help to
beginners. Because, the same thing doesn't work when done through
`vagrant suspend` followed by `vagrant up`.
Due to the directory symlink structure in the dev VM, including the
`~/zulip/contrib_bots/` prefix in the command for running a bot causes
`run.py` to fail with an error.
Aka the current "testing" release, expected to graduate to "stable"
later in 2017.
Fortunately the instructions are very similar to those for
Ubuntu 16.04 and 14.04 -- two packages don't exist, and
those two packages turn out (empirically, on my laptop)
not to be necessary.
Leave most references to "Ubuntu" still just saying "Ubuntu",
on the theory that Debian users will generally follow those
breadcrumbs where they lead and in order to keep lists short.
The integration-guide has now been updated to reflect the recent
decision to rename webhook fixtures from
<webhook_name>/fixtures/<webhook_name>_<event_type>.json to
<webhook_name>/fixtures/<event_type>.json.
The webhook-walkthrough has now been updated to reflect the recent
decision to rename webhook fixtures from
<webhook_name>/fixtures/<webhook_name>_<event_type>.json to
<webhook_name>/fixtures/<event_type>.json.
This commit changes the backend testing framework to run
in parallel mode which is same as --processes=4. If --coverage
is supplied, we enforce serial mode, --processes=1, because
coverage is not compatible with parallel mode at the moment.
Previously, api_key_only_webhook_view passed 3 positional arguments
(request, user_profile, and client) into a function. However, most
of our other auth decorators only pass 2 positional arguments. For
the sake of consistency, we now make api_key_only_webhook_view set
request.client and pass only request and user_profile as positional
arguments.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.
Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.
This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.
Corresponding unit tests were also added.
Fixes#4111.
The integration guide has now been updated to reflect the recent
decision to store webhook fixtures in
zerver/webhooks/<webhook_name>/fixtures/ as opposed to
zerver/fixtures/<webhook_name>/.
The webhook walkthrough has now been updated to reflect the recent
decision to store webhook fixtures in
zerver/webhooks/<webhook_name>/fixtures/ as opposed to
zerver/fixtures/<webhook_name>/.
Update the steps listed in the "Writing a new application
feature" section, based on changes to how realm properties
are created and updated on the backend.
Addresses part of issue #3854.
Rename 'zulip_internal' decorator to 'require_server_admin', add
documentation for 'server_admin', explaining how to give permission
for ./activity page.
Fixes: #1463.
Tweaked this warning because we currently check if there are any
uncommited changes before we reset to a specific pull request.
Changes expanded a bit by tabbott.
It turned out to not be that useful once we added subgroup. The previous
design of the CountStat object also assumed more reuseability of the *_query
strings than what ended up happening.
The filter_args also had some carrying costs:
* It's hard to be confident that filter_args other than the ones explicitly
in our tests would have had expected behavior.
* The filter_args/join_args system is the most complex part of the CountStat
object, and makes understanding the *_query strings unnecessarily
difficult for a new contributor.
Groundwork for allowing stats like "Monthly Active Users".
CountStat.interval is no longer as clean a value as before, so removed it
from views.get_chart_data. It wasn't being used by the frontend anyway.
Removing interval from logger calls in counts.py is not a big loss since we
now include the frequency (which is typically also the interval) in
CountStat.property.
This adds the option '--rerun' to the `test-backend` infrastructure.
It runs the tests that failed during the last 'test-backend' run. It
works by stailing failed test info at var/last_test_failure.json
Cleaned up by Umair Khan and Tim Abbott.
This better sets expectatations for the fact that in Zulip, the
Organization settings UI is available read-only to non-administrator
users.
Tweaked by tabbott to update some additional references.
This is mostly a straight port from bash to Python, but we
rename the coverage option to `--coverage` and we add checks
for being in a venv and being correctly provisioned.
Fixes#4009.
While it's sometimes nice to put a few selectors on the same line,
it is generally better to have a consistent way of formatting our
selectors, and most of our code up until now lists them vertically.
This change fixes the linter to enforce one selector per line, and
it cleans up the places in the CSS where we had multiple selectors
on the same line.
The advantages of one-per-line are as followers:
* cleaner diffs
* easier to see when multiple areas of the app may have the
same format
* less likely to go over 80 cols
* makes it more clear where we have deep nesting in the
individual selectors
* makes it easier for our linting tools to enforce
whitespace violations
This also fixed an old bug where we had ".landing_page h2, h4", which
sets "h4" styles outside of the landing page.
This changes the layout of administration for non-administrators such
that they can view organization settings and emoji settings and
displays everything as readonly unless they have the capability to edit.
For now, we just enabled this for the emoji settings and organization
settings features.
This covers the standard multi-step process for doing large
migrations, as well as other important properties to consider when
writing migrations.
Also documents the new Django 'atomic=False' option.
Fixes#1332.
Our lists of rabbitmq queues was likely to end up out of date, since
there was nothing enforcing that the various lists of queues were
correct or the same as each other.
Add a new section after the Hello World walkthrough for additional detail that
doesn't directly apply to this example. Included are discussions on creating
negative tests and handling custom query parameters.
Remaining integration of the material originally for #3478
Fill in additonal detail following the existing document structure.
Includes authentication, custom streams, negative tests, and types
of test data and fixtures. Also fix typos and reformat to match the
new integration doc style.
Partial integration of the material originally for #3478
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.
Fixes#3363.
This moves do_events_register, fetch_initial_state_data and friends to
a new file.
Modified significantly by tabbott for correctness and to remove unused
imports.
Fixes#3635.
Having just found that a number of our casper tests were buggy because
they were using casper.waitForSelector and fixed those, this seemed
like a good opportunity to update the docs to recommend the selectors
that more faithfully do what developers expect them to.
While I was at it, I tried to make this a bit better organized, though
I think more work could be done on that front.
Breaks out the Hello World example to create a new
webhook-walkthrough.md. Includes minor edits so the two docs
read well. Adds the new page, "Webhook walkthrough", to the TOC.
Fixes#3498
Apparently, the updated version of this has a serious scrolling
performance problem in the left sidebar that basically makes scrolling
in that area unusable.
This reverts commit b683b2d3c3.
- Remove `jquery-mousewheel` from `static/third` and fetch it from npm.
- Upgrade `jquery-mousewheel` to 3.1.6.
- Bump up the `PROVISION_VERSION` to 4.5.
- Change some js code to comply with this `jquery-mousewheel` version.
Part of #1709.
- Remove `underscore.js` from `static/third` and fetch it from `npm`.
- Upgrade `underscore.js` to 1.8.3.
- Bump up the `PROVISION_VERSION` to 4.2.
Part of #1709
- Remove `codepointat` from `static/third` and fetch it from `npm`.
- Upgrade `codepointat` to 0.2.0.
- Bump up the `PROVISION_VERSION` to 4.1.
Part of #1709.
- Remove `winchan.js` from `static/third` and fetch it from `npm`.
- Upgrade `winchan` to 0.2.0.
- Bump up the `PROVISION_VERSION` to 4.0.
Part of #1709.
Before this commit, provisioning was done by executing provision.py,
which printed the log directly to stdout, making debugging harder.
This commit creates a wrapper bash script 'provision' in tools, which
calls 'zulip/scripts/tools/provision_vm.py' (the new location of
provision.py) and prints all the output to
'zulip/var/log/zulip/zulip_provision.log' via 'tee'.
Travis tests and docs have been modified accordingly.
This reverts commit 7bf10ec74f.
Apparently, SockJS 1.1.1 is broken with the browser used in our legacy
desktop app, resulting in messages being silently not sent.
This adds some configuration options to settings.py, namely
PASSWORD_MIN_LENGTH and PASSWORD_MIN_QUALITY, which control
when the frontend validator invalidates the password.
Closes#2628
Robert Hönig
Tim Abbott
Supermanu
Sarah
rht
Hobson Lane
Vaida Plankyte
Umair Khan
David Taylor
Greg Price
Aaron Tse
Steve Howell
Jack Zhang
Pweaver (Paul Weaver)
Steve Howell
Jason Michalski
Cynthia Lin
Rishi Gupta
Shubham Gupta
Vishnu Ks
Durga Akhil Mundroy
Rohitt Vashishtha
Harshit Bansal
Aditya Bansal
Eeshan Garg
Cory Lynch
vaibhav
Vaida Plankyte
Rafid Aslam
Feorlen
Joshua Pan
Saket Kumar
neiljp
Abhijeet Kaur
Ian Winter
Reid Barton
Mahim Goyal
Theodore Chen
Jeremy Bowman
Elliott Jin
Carey Metcalfe
jersou
Josh Mandel
Diego Berrocal
vbNETonIce
Emilio Aburto L
Maxim Averin
Raghav Jajodia
K.Kanakhin
Adarsh S
Sumana Harihareswara
Tomasz Kolek
Luis Saul Trinidad
Yago González
Moritz Neeb
Akash Kothawale
sonali0901
Brock Whittaker
Sourav Badami
Alicja Raszkowska
saisrivathsa
Kouhei Sutou
Yago González
Eitan Adler
Jackson
JefftheBest1
Bojidar Marinov
Tommy Ip
Alex Morozov
Alya Abbott
Rohan Tibrewal