+ + diff --git a/templates/web/harrogate/report/new/councils_extra_text.html b/templates/web/harrogate/report/new/councils_extra_text.html new file mode 100644 index 000000000..92dc93452 --- /dev/null +++ b/templates/web/harrogate/report/new/councils_extra_text.html @@ -0,0 +1,10 @@ +
+

+NB: though we will forward your request as appropriate, +reports for problems managed by North Yorkshire County Council are not shown on this map. +View all problems in that area on +FixMyStreet.com. +

+
+ +Do not use this to report emergencies outside of working hours. diff --git a/templates/web/harrogate/static/privacy.html b/templates/web/harrogate/static/privacy.html new file mode 100755 index 000000000..0107d0873 --- /dev/null +++ b/templates/web/harrogate/static/privacy.html @@ -0,0 +1,56 @@ +[% INCLUDE 'header.html', + title = loc('Frequently Asked Questions'), + bodyclass = 'twothirdswidthpage' %] + +
+ +
+ +

Privacy, cookies, and third party services

+ +

Our use of your data, cookies, and external services: what you +should know, and how to opt out if you want to.

+ +

Privacy

+ +
+
Who gets to see my email address?
+
+

If you submit a problem, your details, and details of the problem, will be + submitted to Harrogate Borough Council, or North Yorkshire County Council, as + appropriate. Other than the council, only people we authorise to view the + FixMyStreet administration interface will be able to see your email address and + they will never use it for anything other than to help administer + FixMyStreet.

+

We will never give or sell your email address to anyone else, unless we are + obliged to by law. Your name will not be published anywhere unless you let us.

+
+ +
What emails will you send to me?
+
We will email you when we have received your report, and when it has + been investigated and actioned. We will only send you emails that + relate to an issue you have reported.
+ +
+
+ +

Cookies

+ +

To make our service easier or more useful, we sometimes place small data +files on your computer or mobile phone, known as cookies; many websites do +this. We use this information to, for example, remember you have logged in so +you don't need to do that on every page. Below, we list the cookies and +services that this site can use. + + + + +
NameTypical ContentExpires
fixmystreet_app_sessionA random unique identifierWhen browser is closed, or four weeks if “Keep me signed in” is ticked
+ +[% INCLUDE 'footer.html' pagefooter = 'yes' %] diff --git a/web/cobrands/harrogate/HBCLogo-cropped.png b/web/cobrands/harrogate/HBCLogo-cropped.png new file mode 100755 index 000000000..4421820d2 Binary files /dev/null and b/web/cobrands/harrogate/HBCLogo-cropped.png differ diff --git a/web/cobrands/harrogate/HBLogo.png b/web/cobrands/harrogate/HBLogo.png new file mode 100755 index 000000000..03cf2b4f5 Binary files /dev/null and b/web/cobrands/harrogate/HBLogo.png differ diff --git a/web/cobrands/harrogate/_colours.scss b/web/cobrands/harrogate/_colours.scss index 2b9714835..a79b7acfb 100644 --- a/web/cobrands/harrogate/_colours.scss +++ b/web/cobrands/harrogate/_colours.scss @@ -18,5 +18,5 @@ $base_fg: #1a1a1a; /* Following seem to be required in layout.scss */ $map_nav_bg: #222; -$nav_fg: #fff; +$nav_fg: #000; $nav_fg_hover: #444; diff --git a/web/cobrands/harrogate/base.scss b/web/cobrands/harrogate/base.scss index 94dd39ee7..eccb1a84f 100644 --- a/web/cobrands/harrogate/base.scss +++ b/web/cobrands/harrogate/base.scss @@ -11,10 +11,11 @@ body.frontpage #site-logo, #site-logo { display: block; - background: url("/cobrands/harrogate/HBCLogo-mobile.png") 0 0 no-repeat; margin: 2px; padding-left: 10px; position: static; + + background: url("/cobrands/harrogate/HBCLogo-mobile.png") 0 0 no-repeat; width: 106px; height: 37px; } @@ -22,21 +23,43 @@ body.frontpage #site-logo, #main-nav ul#main-menu li { // blue button background, darker if selected - background: $primary; - &.menu-item-selected { background: $primary_dark; } - - @include border-radius(1em); - padding: 0.25em; - margin: 0.25em; - border: solid 1px black; + background: #ededed; + &.menu-item-selected { + background: $primary; + span { + color: #fff; + } + } span { color: $nav_fg; } } +#main-nav ul#main-menu li, +body.mappage #main-nav ul#main-menu li, +{ + padding-left: 0.25em; + padding-right: 0.25em; + padding-top: 0; + padding-bottom: 0; + + a { padding: 0 } +} + label[for=form_INFO_TEXT] { background: yellow; border: solid 1px black; padding: 1em; } + +#site-header { + background: none; +} + + +#council_responsibilities_note { + background-color: lighten($primary, 30%); + padding: 0.5em; + font-size: 0.8em; +} diff --git a/web/cobrands/harrogate/layout.scss b/web/cobrands/harrogate/layout.scss index 0e76bb111..480a07f5d 100644 --- a/web/cobrands/harrogate/layout.scss +++ b/web/cobrands/harrogate/layout.scss @@ -2,24 +2,184 @@ @import "../sass/layout"; @media only screen and (min-width: 48em) { + #site-logo, + .ie6 #site-logo, + body.frontpage #site-logo, .ie6 body.frontpage #site-logo, + + body.authpage #site-logo, + .ie6 body.authpage #site-logo, + body.twothirdswidthpage #site-logo, .ie6 body.twothirdswidthpage #site-logo { position: relative; - top: 5px; + top: 1.2em; margin: 0; - margin-left: 2.25em; padding: 0; background: url("/cobrands/harrogate/HBCLogo.png") 0 0 no-repeat; width: 227px; height: 122px; } + + body.frontpage #site-header, + .ie6 body.frontpage #site-header, + + body.twothirdswidthpage #site-header, + .ie6 body.twothirdswidthpage #site-header, + + body.authpage #site-header, + .ie6 body.authpage #site-header, + + body.mappage #site-header, + .ie6 body.mappage #site-header, + body.mappage .nav-wrapper-2, + .ie6 body.mappage .nav-wrapper-2 + { + height: 10em; + } + + #fms_pan_zoom { + top: 10.75em!important; + } +} + +@media only screen and (min-width: 48em) { + body.twothirdswidthpage .container .content footer .tablewrapper a:link, + body.twothirdswidthpage .container .content footer .tablewrapper a:visited, + body.fullwidthpage .container .content footer .tablewrapper a:link, + body.fullwidthpage .container .content footer .tablewrapper a:visited { + color: white; + } + + body.mappage #user-meta+.container .content { + padding-top: 1.5em; + } +} + +body.fullwidthpage #main-nav, +body.frontpage #main-nav, +body.twothirdswidthpage #main-nav, +body.authpage #main-nav +{ + margin-top: 3.2em; +} + +body.mappage #main-nav, +{ + // TODO would be better to edit containers to figure out where missing em is defined + margin-top: 2.2em; +} + +body.fullwidthpage .container .content, +body.twothirdswidthpage .container .content { + margin: 0; + padding-left: 0; + padding-right: 0; +} + +@media only screen and (min-width: 48em) { + body.fullwidthpage .container .content { + margin-top: 3em; // push far enough for the sign-out link to also be visible + } +} + +body.twothirdswidthpage .content aside { + margin-top: 2em; + @include box-shadow(none); +} + + +body.twothirdswidthpage form.full-width { + margin: 0; +} + +body.mappage #main-nav { + ul#main-menu { + padding: 1em 0; + } +} + +body.mappage #main-nav ul#main-menu li a, body.mappage #main-nav ul#main-menu li span { + padding: 0.75em; +} + +body.mappage #main-nav ul#main-menu li a.report-a-problem-btn, +#main-nav ul#main-menu li a.report-a-problem-btn { + margin: 0; + padding: 0.75em; +} +#main-nav ul#main-menu li a.report-a-problem-btn { + &:hover { + background: #444; + color: #fff; + @include border-radius(0em); + } +} + +#main-nav { + ul#main-menu { + li { + a.report-a-problem-btn { + color: #000; + background: #ededed; + } + } + } +} + +body.mappage .nav-wrapper .nav-wrapper-2 { + background: #fff; + border-bottom: solid 4px $primary; +} + +.nav-wrapper .nav-wrapper-2 { + border-top: 0px; } -#main-nav ul#main-menu li { - span { - color: $nav_fg; +#front-main { + background: none; + padding: 0; + + #front-main-container { + background: $primary; + padding: 2em 0; + margin-bottom: 1.2em; + @include border-radius(12px); + + #postcode-intro { + margin: 0 2em; + } + } + +} + +body.frontpage #user-meta p { + top: -2.4em; +} + +body.mappage #user-meta { + top: 2em; + p { + background: $primary; + border-color: $primary_dark; + } +} + +body.mappage .content { + margin-top: 4em; +} + + +body.twothirdswidthpage .container .content footer, +body.fullwidthpage .container .content footer +{ + .tablewrapper { + width: 100%; + background: $primary; + @include border-radius(12px); + margin-bottom: 2em; + } } -- cgit v1.2.3 From b86b78c1f61e31ab59940d5c82b79961bccb56f9 Mon Sep 17 00:00:00 2001 From: Hakim Cassimally Date: Thu, 8 Jan 2015 14:50:49 +0000 Subject: [Harrogate] update email for live deployment --- perllib/FixMyStreet/Cobrand/Harrogate.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/perllib/FixMyStreet/Cobrand/Harrogate.pm b/perllib/FixMyStreet/Cobrand/Harrogate.pm index 7249bcece..6bcc2f227 100644 --- a/perllib/FixMyStreet/Cobrand/Harrogate.pm +++ b/perllib/FixMyStreet/Cobrand/Harrogate.pm @@ -56,7 +56,7 @@ Can run with a script or command line like: =cut sub temp_email_to_update { - return 'test@example.com'; + return 'CustomerServices@harrogate.gov.uk'; } sub temp_update_contacts { -- cgit v1.2.3 From 0403fb7e2abcac670039372181c870a8c2815d2a Mon Sep 17 00:00:00 2001 From: Hakim Cassimally Date: Fri, 9 Jan 2015 14:54:45 +0000 Subject: [EastSussex] disable questionnaires at ESCC request today --- perllib/FixMyStreet/Cobrand/EastSussex.pm | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/perllib/FixMyStreet/Cobrand/EastSussex.pm b/perllib/FixMyStreet/Cobrand/EastSussex.pm index 5447545e7..5aabae449 100644 --- a/perllib/FixMyStreet/Cobrand/EastSussex.pm +++ b/perllib/FixMyStreet/Cobrand/EastSussex.pm @@ -119,5 +119,9 @@ sub pin_colour { return $open_states->{ $p->state } ? 'yellow' : 'green'; } +sub send_questionnaires { + return 0; +} + 1; -- cgit v1.2.3 From 14384f023d897e40a6f02a58eea0f3d84c6584d8 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Fri, 9 Jan 2015 17:40:53 +0000 Subject: Revert most of "Experiment, no pins by default." This reverts most of commit 71b400a1f5991b9a693095c417aa61e274a5215d, leaving the skip-this-step CSS changes and the tweaks to the /ajax JSON output. --- .../web/fixmystreet.com/around/_report_banner.html | 11 ------ .../web/fixmystreet.com/around/tabbed_lists.html | 46 ---------------------- templates/web/fixmystreet.com/header_extra.html | 21 ---------- web/cobrands/fixmystreet/fixmystreet.js | 15 +------ web/cobrands/fixmystreet/layout.scss | 9 ----- web/js/map-OpenLayers.js | 4 -- 6 files changed, 2 insertions(+), 104 deletions(-) delete mode 100755 templates/web/fixmystreet.com/around/_report_banner.html delete mode 100644 templates/web/fixmystreet.com/around/tabbed_lists.html diff --git a/templates/web/fixmystreet.com/around/_report_banner.html b/templates/web/fixmystreet.com/around/_report_banner.html deleted file mode 100755 index 9d0d9bdfe..000000000 --- a/templates/web/fixmystreet.com/around/_report_banner.html +++ /dev/null @@ -1,11 +0,0 @@ -[%# Identical to parent, just with variant0s %] - -

- [% loc( 'Click map to report a problem' ) %] - [% IF c.cobrand.moniker == 'bromley' %] - Yellow pins show existing reports - [% END %] -

- - [% loc("Can't see the map? Skip this step") %] - diff --git a/templates/web/fixmystreet.com/around/tabbed_lists.html b/templates/web/fixmystreet.com/around/tabbed_lists.html deleted file mode 100644 index 6a3cdf8a9..000000000 --- a/templates/web/fixmystreet.com/around/tabbed_lists.html +++ /dev/null @@ -1,46 +0,0 @@ - - - - - -
-
-

[% loc( 'Click the map to
report a problem' ) %]

- - - [% loc("Can't see the map? Skip this step") %] - - -

Don’t worry if it’s not in precisely the right place,
- you can move the pin afterwards.

-
-
- -
-
-
    - [% INCLUDE "around/on_map_list_items.html" %] -
-
-
- -
-
    - [% INCLUDE "around/around_map_list_items.html" %] -
-
- - diff --git a/templates/web/fixmystreet.com/header_extra.html b/templates/web/fixmystreet.com/header_extra.html index 65f88be14..27ffc12a1 100644 --- a/templates/web/fixmystreet.com/header_extra.html +++ b/templates/web/fixmystreet.com/header_extra.html @@ -1,24 +1,3 @@ -[%# We are conducting an A/B experiment on the first step of reporting process %] -[% IF page == 'around' %] - - - -[% END %] - [% INCLUDE 'tracking_code.html' %] diff --git a/web/cobrands/fixmystreet/fixmystreet.js b/web/cobrands/fixmystreet/fixmystreet.js index 4e7315017..fdecee9db 100644 --- a/web/cobrands/fixmystreet/fixmystreet.js +++ b/web/cobrands/fixmystreet/fixmystreet.js @@ -46,17 +46,6 @@ function tabs(elem, indirect) { //hide / show the right tab $('.tab.open').hide().removeClass('open'); $(target).show().addClass('open'); - - // Clicking on tabs to show/hide pins, content experiment - if (typeof(variation) !== 'undefined' && variation == 1) { - if (target == '#current_near_tab' && !fixmystreet.markers.getVisibility()) { - $('#hide_pins_link').click(); - } - if (target == '#reporting' && fixmystreet.markers.getVisibility()) { - $('#hide_pins_link').click(); - } - } - } } @@ -187,8 +176,8 @@ $(function(){ * Tabs */ //make initial tab active - $('.tab-nav a:visible').first().addClass('active'); - $('.tab:visible').first().addClass('open'); + $('.tab-nav a').first().addClass('active'); + $('.tab').first().addClass('open'); //hide other tabs $('.tab').not('.open').hide(); diff --git a/web/cobrands/fixmystreet/layout.scss b/web/cobrands/fixmystreet/layout.scss index 21de4f271..ded1ed911 100644 --- a/web/cobrands/fixmystreet/layout.scss +++ b/web/cobrands/fixmystreet/layout.scss @@ -92,15 +92,6 @@ body.mappage { @include background(linear-gradient(#000, #222 10%, #222 90%, #000)); } } - - #reporting { - padding: 1em; - text-align: center; - - h1 { - line-height: 1.2em; - } - } } #main-nav { diff --git a/web/js/map-OpenLayers.js b/web/js/map-OpenLayers.js index 63625efb8..c7bfef2b0 100644 --- a/web/js/map-OpenLayers.js +++ b/web/js/map-OpenLayers.js @@ -227,10 +227,6 @@ function fixmystreet_onload() { } }); - if (fixmystreet.initial_hide_pins) { - $('#hide_pins_link').click(); - } - $('#all_pins_link').click(function(e) { e.preventDefault(); fixmystreet.markers.setVisibility(true); -- cgit v1.2.3 From 4ae8d597a80d91084dcdcd999d480f41a57c70d1 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Fri, 9 Jan 2015 17:58:43 +0000 Subject: Use HTTPS OpenStreetMap tiles. --- perllib/FixMyStreet/Map/OSM.pm | 8 ++++---- web/js/map-OpenStreetMap.js | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/perllib/FixMyStreet/Map/OSM.pm b/perllib/FixMyStreet/Map/OSM.pm index 74af0e9f3..82ddd293b 100644 --- a/perllib/FixMyStreet/Map/OSM.pm +++ b/perllib/FixMyStreet/Map/OSM.pm @@ -29,10 +29,10 @@ sub map_tiles { my ( $x, $y, $z ) = ( $params{x_tile}, $params{y_tile}, $params{zoom_act} ); my $tile_url = $self->base_tile_url(); return [ - "http://a.$tile_url/$z/" . ($x - 1) . "/" . ($y - 1) . ".png", - "http://b.$tile_url/$z/$x/" . ($y - 1) . ".png", - "http://c.$tile_url/$z/" . ($x - 1) . "/$y.png", - "http://$tile_url/$z/$x/$y.png", + "https://a.$tile_url/$z/" . ($x - 1) . "/" . ($y - 1) . ".png", + "https://b.$tile_url/$z/$x/" . ($y - 1) . ".png", + "https://c.$tile_url/$z/" . ($x - 1) . "/$y.png", + "https://a.$tile_url/$z/$x/$y.png", ]; } diff --git a/web/js/map-OpenStreetMap.js b/web/js/map-OpenStreetMap.js index fc0afedc3..7ef222da7 100644 --- a/web/js/map-OpenStreetMap.js +++ b/web/js/map-OpenStreetMap.js @@ -8,7 +8,7 @@ function set_map_config(perm) { //new OpenLayers.Control.LayerSwitcher(), new OpenLayers.Control.Navigation(), new OpenLayers.Control.PermalinkFMS(permalink_id), - new OpenLayers.Control.PermalinkFMSz('osm_link', 'http://www.openstreetmap.org/'), + new OpenLayers.Control.PermalinkFMSz('osm_link', 'https://www.openstreetmap.org/'), new OpenLayers.Control.PanZoomFMS({id: 'fms_pan_zoom' }) ]; } @@ -31,9 +31,9 @@ OpenLayers.Layer.OSM.Mapnik = OpenLayers.Class(OpenLayers.Layer.OSM, { */ initialize: function(name, options) { var url = [ - "http://a.tile.openstreetmap.org/${z}/${x}/${y}.png", - "http://b.tile.openstreetmap.org/${z}/${x}/${y}.png", - "http://c.tile.openstreetmap.org/${z}/${x}/${y}.png" + "https://a.tile.openstreetmap.org/${z}/${x}/${y}.png", + "https://b.tile.openstreetmap.org/${z}/${x}/${y}.png", + "https://c.tile.openstreetmap.org/${z}/${x}/${y}.png" ]; options = OpenLayers.Util.extend({ /* Below line added to OSM's file in order to allow minimum zoom level */ -- cgit v1.2.3 From 676181084d88b0ec94d42521162b4571cb0c0552 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Fri, 9 Jan 2015 22:55:08 +0000 Subject: Use same handling for cron and non-cron email. This means that e.g. SMTP authentication is used when set up by all emails, not just non-cron ones. Fixes #988. --- bin/send-comments | 1 - bin/zurich-overdue-alert | 5 +--- conf/general.yml-example | 2 +- perllib/FixMyStreet/App.pm | 32 +++++++++++++++-------- perllib/FixMyStreet/App/Controller/Contact.pm | 2 ++ perllib/FixMyStreet/App/Model/EmailSend.pm | 4 +-- perllib/FixMyStreet/DB/ResultSet/AlertType.pm | 4 +-- perllib/FixMyStreet/DB/ResultSet/Problem.pm | 6 ++--- perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm | 4 +-- perllib/FixMyStreet/EmailSend.pm | 9 ------- perllib/FixMyStreet/EmailSend/ContactEmail.pm | 9 +++++++ perllib/FixMyStreet/EmailSend/DoNotReply.pm | 9 +++++++ perllib/FixMyStreet/SendReport.pm | 2 ++ perllib/FixMyStreet/SendReport/Email.pm | 32 +++++++++++------------ perllib/FixMyStreet/SendReport/EmptyHomes.pm | 11 +++----- perllib/FixMyStreet/SendReport/NI.pm | 5 +--- perllib/FixMyStreet/SendReport/Zurich.pm | 2 +- t/app/model/problem.t | 2 +- t/app/sendreport/email.t | 4 +-- 19 files changed, 75 insertions(+), 70 deletions(-) delete mode 100644 perllib/FixMyStreet/EmailSend.pm create mode 100644 perllib/FixMyStreet/EmailSend/ContactEmail.pm create mode 100644 perllib/FixMyStreet/EmailSend/DoNotReply.pm diff --git a/bin/send-comments b/bin/send-comments index 03adfb095..2192f674c 100755 --- a/bin/send-comments +++ b/bin/send-comments @@ -21,7 +21,6 @@ use FixMyStreet::App; use Utils; use mySociety::Config; -use mySociety::EmailUtil; use Open311; diff --git a/bin/zurich-overdue-alert b/bin/zurich-overdue-alert index e7c23b493..c09aef1e2 100755 --- a/bin/zurich-overdue-alert +++ b/bin/zurich-overdue-alert @@ -62,16 +62,14 @@ sub send_alert { admin_url => $cobrand->admin_base_url, }; - my $env_to = [ $body_email ]; my $to = [ [ $body_email, $body->name ] ]; if ( $include_parent ) { my $parent = $body->parent; my $parent_email = $parent->endpoint; - push @$env_to, $parent_email; push @$to, [ $parent_email, $parent->name ]; } - my $result = FixMyStreet::App->send_email_cron( + FixMyStreet::App->send_email_cron( { _template_ => $template, _parameters_ => $h, @@ -79,7 +77,6 @@ sub send_alert { From => [ FixMyStreet->config('CONTACT_EMAIL'), FixMyStreet->config('CONTACT_NAME') ], }, FixMyStreet->config('CONTACT_EMAIL'), - $env_to, $nomail ); } diff --git a/conf/general.yml-example b/conf/general.yml-example index fc20a90ea..9770ade86 100644 --- a/conf/general.yml-example +++ b/conf/general.yml-example @@ -37,7 +37,7 @@ DO_NOT_REPLY_EMAIL: 'do-not-reply@example.org' # Whether this is a development site or not. It will mean e.g. templates/ # CSS modified times aren't cached STAGING_SITE: 1 -# Normally, a staging site will route all reports to CONTACT_EMAIL on a +# Normally, a staging site will route all reports to the reporter on a # development site (when STAGING_SITE is 1), to guard against sending fake # reports to live places. Set this to 1 if you want a dev site to route # reports as normal. diff --git a/perllib/FixMyStreet/App.pm b/perllib/FixMyStreet/App.pm index e5e483937..fd70ffda8 100644 --- a/perllib/FixMyStreet/App.pm +++ b/perllib/FixMyStreet/App.pm @@ -10,7 +10,6 @@ use FixMyStreet; use FixMyStreet::Cobrand; use Memcached; use mySociety::Email; -use mySociety::EmailUtil; use mySociety::Random qw(random_bytes); use FixMyStreet::Map; @@ -348,9 +347,19 @@ sub send_email { } sub send_email_cron { - my ( $c, $params, $env_from, $env_to, $nomail, $cobrand, $lang_code ) = @_; - - return 1 if $c->is_abuser( $env_to ); + my ( $c, $params, $env_from, $nomail, $cobrand, $lang_code ) = @_; + + my $first_to; + if (ref($params->{To}) eq 'ARRAY') { + if (ref($params->{To}[0]) eq 'ARRAY') { + $first_to = $params->{To}[0][0]; + } else { + $first_to = $params->{To}[0]; + } + } else { + $first_to = $params->{To}; + } + return 1 if $c->is_abuser($first_to); $params->{'Message-ID'} = sprintf('', time(), unpack('h*', random_bytes(5, 1)), FixMyStreet->config('EMAIL_DOMAIN') @@ -387,15 +396,16 @@ sub send_email_cron { $params->{_line_indent} = ''; my $email = mySociety::Locale::in_gb_locale { mySociety::Email::construct_email($params) }; - if ( FixMyStreet->test_mode ) { - my $sender = Email::Send->new({ mailer => 'Test' }); - $sender->send( $email ); - return 0; - } elsif (!$nomail) { - return mySociety::EmailUtil::send_email( $email, $env_from, @$env_to ); - } else { + if ($nomail) { print $email; return 1; # Failure + } else { + my %model_args; + if (!FixMyStreet->test_mode && $env_from eq FixMyStreet->config('CONTACT_EMAIL')) { + $model_args{mailer} = 'FixMyStreet::EmailSend::ContactEmail'; + } + my $result = $c->model('EmailSend', %model_args)->send($email); + return $result ? 0 : 1; } } diff --git a/perllib/FixMyStreet/App/Controller/Contact.pm b/perllib/FixMyStreet/App/Controller/Contact.pm index 5a51c8494..f48518d77 100644 --- a/perllib/FixMyStreet/App/Controller/Contact.pm +++ b/perllib/FixMyStreet/App/Controller/Contact.pm @@ -4,6 +4,8 @@ use namespace::autoclean; BEGIN { extends 'Catalyst::Controller'; } +use mySociety::EmailUtil; + =head1 NAME FixMyStreet::App::Controller::Contact - Catalyst Controller diff --git a/perllib/FixMyStreet/App/Model/EmailSend.pm b/perllib/FixMyStreet/App/Model/EmailSend.pm index 7f130c26d..475026267 100644 --- a/perllib/FixMyStreet/App/Model/EmailSend.pm +++ b/perllib/FixMyStreet/App/Model/EmailSend.pm @@ -1,5 +1,5 @@ package FixMyStreet::App::Model::EmailSend; -use base 'Catalyst::Model::Adaptor'; +use base 'Catalyst::Model::Factory'; use strict; use warnings; @@ -54,7 +54,7 @@ elsif ( my $smtp_host = FixMyStreet->config('SMTP_SMARTHOST') ) { push @$mailer_args, username => $username, password => $password if $username && $password; $args = { - mailer => 'FixMyStreet::EmailSend', + mailer => 'FixMyStreet::EmailSend::DoNotReply', mailer_args => $mailer_args, }; } diff --git a/perllib/FixMyStreet/DB/ResultSet/AlertType.pm b/perllib/FixMyStreet/DB/ResultSet/AlertType.pm index 5bed95811..b704fa7dd 100644 --- a/perllib/FixMyStreet/DB/ResultSet/AlertType.pm +++ b/perllib/FixMyStreet/DB/ResultSet/AlertType.pm @@ -5,7 +5,6 @@ use strict; use warnings; use mySociety::DBHandle qw(dbh); -use mySociety::EmailUtil; use mySociety::Gaze; use mySociety::Locale; use mySociety::MaPit; @@ -262,13 +261,12 @@ sub _send_aggregated_alert_email(%) { To => $data{alert_email}, }, $sender, - [ $data{alert_email} ], 0, $cobrand, $data{lang} ); - if ($result == mySociety::EmailUtil::EMAIL_SUCCESS) { + unless ($result) { $token->insert(); } else { print "Failed to send alert $data{alert_id}!"; diff --git a/perllib/FixMyStreet/DB/ResultSet/Problem.pm b/perllib/FixMyStreet/DB/ResultSet/Problem.pm index d3c016be6..a84a309ee 100644 --- a/perllib/FixMyStreet/DB/ResultSet/Problem.pm +++ b/perllib/FixMyStreet/DB/ResultSet/Problem.pm @@ -8,7 +8,6 @@ use CronFns; use Utils; use mySociety::Config; -use mySociety::EmailUtil; use mySociety::MaPit; use FixMyStreet::App; @@ -438,7 +437,7 @@ sub send_reports { } } - if ($result == mySociety::EmailUtil::EMAIL_SUCCESS) { + unless ($result) { $row->update( { whensent => \'ms_current_timestamp()', lastupdate => \'ms_current_timestamp()', @@ -501,7 +500,7 @@ sub _send_report_sent_email { my $template = FixMyStreet->get_email_template($row->cobrand, $row->lang, 'confirm_report_sent.txt'); - my $result = FixMyStreet::App->send_email_cron( + FixMyStreet::App->send_email_cron( { _template_ => $template, _parameters_ => $h, @@ -509,7 +508,6 @@ sub _send_report_sent_email { From => mySociety::Config::get('CONTACT_EMAIL'), }, mySociety::Config::get('CONTACT_EMAIL'), - [ $row->user->email ], $nomail, $cobrand ); diff --git a/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm b/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm index b7af9e60e..646967bfc 100644 --- a/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm +++ b/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm @@ -5,7 +5,6 @@ use strict; use warnings; use Encode; use Utils; -use mySociety::EmailUtil; sub send_questionnaires { my ( $rs, $params ) = @_; @@ -103,11 +102,10 @@ sub send_questionnaires_period { From => [ $sender, $sender_name ], }, $sender, - [ $row->user->email ], $params->{nomail}, $cobrand ); - if ($result == mySociety::EmailUtil::EMAIL_SUCCESS) { + unless ($result) { print " ...success\n" if $params->{verbose}; $row->update(); $token->insert(); diff --git a/perllib/FixMyStreet/EmailSend.pm b/perllib/FixMyStreet/EmailSend.pm deleted file mode 100644 index 8b6eed462..000000000 --- a/perllib/FixMyStreet/EmailSend.pm +++ /dev/null @@ -1,9 +0,0 @@ -package FixMyStreet::EmailSend; -use base Email::Send::SMTP; - -sub get_env_sender { - my $sender = FixMyStreet->config('DO_NOT_REPLY_EMAIL'); - return $sender; -} - -1; diff --git a/perllib/FixMyStreet/EmailSend/ContactEmail.pm b/perllib/FixMyStreet/EmailSend/ContactEmail.pm new file mode 100644 index 000000000..28bcc983b --- /dev/null +++ b/perllib/FixMyStreet/EmailSend/ContactEmail.pm @@ -0,0 +1,9 @@ +package FixMyStreet::EmailSend::ContactEmail; +use base Email::Send::SMTP; + +sub get_env_sender { + my $sender = FixMyStreet->config('CONTACT_EMAIL'); + return $sender; +} + +1; diff --git a/perllib/FixMyStreet/EmailSend/DoNotReply.pm b/perllib/FixMyStreet/EmailSend/DoNotReply.pm new file mode 100644 index 000000000..d1368f00f --- /dev/null +++ b/perllib/FixMyStreet/EmailSend/DoNotReply.pm @@ -0,0 +1,9 @@ +package FixMyStreet::EmailSend::DoNotReply; +use base Email::Send::SMTP; + +sub get_env_sender { + my $sender = FixMyStreet->config('DO_NOT_REPLY_EMAIL'); + return $sender; +} + +1; diff --git a/perllib/FixMyStreet/SendReport.pm b/perllib/FixMyStreet/SendReport.pm index 5087c7ead..40e76ef72 100644 --- a/perllib/FixMyStreet/SendReport.pm +++ b/perllib/FixMyStreet/SendReport.pm @@ -10,6 +10,7 @@ use Module::Pluggable has 'body_config' => ( is => 'rw', isa => 'HashRef', default => sub { {} } ); has 'bodies' => ( is => 'rw', isa => 'ArrayRef', default => sub { [] } ); has 'to' => ( is => 'rw', isa => 'ArrayRef', default => sub { [] } ); +has 'bcc' => ( is => 'rw', isa => 'ArrayRef', default => sub { [] } ); has 'success' => ( is => 'rw', isa => 'Bool', default => 0 ); has 'error' => ( is => 'rw', isa => 'Str', default => '' ); has 'unconfirmed_counts' => ( 'is' => 'rw', isa => 'HashRef', default => sub { {} } ); @@ -44,6 +45,7 @@ sub reset { $self->bodies( [] ); $self->body_config( {} ); $self->to( [] ); + $self->bcc( [] ); } sub add_body { diff --git a/perllib/FixMyStreet/SendReport/Email.pm b/perllib/FixMyStreet/SendReport/Email.pm index 19c6405d2..797b41e91 100644 --- a/perllib/FixMyStreet/SendReport/Email.pm +++ b/perllib/FixMyStreet/SendReport/Email.pm @@ -4,11 +4,8 @@ use Moose; BEGIN { extends 'FixMyStreet::SendReport'; } -use mySociety::EmailUtil; - sub build_recipient_list { my ( $self, $row, $h ) = @_; - my %recips; my $all_confirmed = 1; foreach my $body ( @{ $self->bodies } ) { @@ -49,12 +46,10 @@ sub build_recipient_list { } for my $email ( @emails ) { push @{ $self->to }, [ $email, $body_name ]; - $recips{$email} = 1; } } - return () unless $all_confirmed; - return keys %recips; + return $all_confirmed && @{$self->to}; } sub get_template { @@ -76,34 +71,36 @@ sub send { my $self = shift; my ( $row, $h ) = @_; - my @recips = $self->build_recipient_list( $row, $h ); + my $recips = $self->build_recipient_list( $row, $h ); # on a staging server send emails to ourselves rather than the bodies if (mySociety::Config::get('STAGING_SITE') && !mySociety::Config::get('SEND_REPORTS_ON_STAGING') && !FixMyStreet->test_mode) { - @recips = ( $row->user->email ); + $recips = 1; + @{$self->to} = [ $row->user->email, $self->to->[0][1] || $row->name ]; } - unless ( @recips ) { + unless ($recips) { $self->error( 'No recipients' ); return 1; } my ($verbose, $nomail) = CronFns::options(); my $cobrand = FixMyStreet::Cobrand->get_class_for_moniker($row->cobrand)->new(); + my $params = { + _template_ => $self->get_template( $row ), + _parameters_ => $h, + To => $self->to, + From => $self->send_from( $row ), + }; + $params->{Bcc} = $self->bcc if @{$self->bcc}; my $result = FixMyStreet::App->send_email_cron( - { - _template_ => $self->get_template( $row ), - _parameters_ => $h, - To => $self->to, - From => $self->send_from( $row ), - }, + $params, mySociety::Config::get('CONTACT_EMAIL'), - \@recips, $nomail, $cobrand ); - if ( $result == mySociety::EmailUtil::EMAIL_SUCCESS ) { + unless ($result) { $self->success(1); } else { $self->error( 'Failed to send email' ); @@ -143,4 +140,5 @@ sub _get_district_for_contact { ($district) = keys %$district; return $district; } + 1; diff --git a/perllib/FixMyStreet/SendReport/EmptyHomes.pm b/perllib/FixMyStreet/SendReport/EmptyHomes.pm index 4bae6af46..ce69aaac3 100644 --- a/perllib/FixMyStreet/SendReport/EmptyHomes.pm +++ b/perllib/FixMyStreet/SendReport/EmptyHomes.pm @@ -9,7 +9,6 @@ BEGIN { extends 'FixMyStreet::SendReport::Email'; } sub build_recipient_list { my ( $self, $row, $h ) = @_; - my %recips; my $all_confirmed = 1; foreach my $body ( @{ $self->bodies } ) { @@ -31,24 +30,22 @@ sub build_recipient_list { } push @{ $self->to }, [ $body_email, $body->name ]; - $recips{$body_email} = 1; my $area_info = mySociety::MaPit::call('area', $body->body_areas->first->area_id); my $country = $area_info->{country}; if ($country eq 'W') { - $recips{ 'wales@' . mySociety::Config::get('EMAIL_DOMAIN') } = 1; + push @{$self->bcc}, 'wales@' . mySociety::Config::get('EMAIL_DOMAIN'); } elsif ($country eq 'S') { - $recips{ 'scotland@' . mySociety::Config::get('EMAIL_DOMAIN') } = 1; + push @{$self->bcc}, 'scotland@' . mySociety::Config::get('EMAIL_DOMAIN'); } else { - $recips{ 'eha@' . mySociety::Config::get('EMAIL_DOMAIN') } = 1; + push @{$self->bcc}, 'eha@' . mySociety::Config::get('EMAIL_DOMAIN'); } } # Set address email parameter from added data $h->{address} = $row->extra->{address}; - return () unless $all_confirmed; - return keys %recips; + return $all_confirmed && @{$self->to}; } sub get_template { diff --git a/perllib/FixMyStreet/SendReport/NI.pm b/perllib/FixMyStreet/SendReport/NI.pm index e0ea24f9c..c60643566 100644 --- a/perllib/FixMyStreet/SendReport/NI.pm +++ b/perllib/FixMyStreet/SendReport/NI.pm @@ -6,7 +6,6 @@ BEGIN { extends 'FixMyStreet::SendReport::Email'; } sub build_recipient_list { my ( $self, $row, $h ) = @_; - my %recips; my $all_confirmed = 1; foreach my $body ( @{ $self->bodies } ) { @@ -30,11 +29,9 @@ sub build_recipient_list { $row->external_body( 'Roads Service' ); } push @{ $self->to }, [ $email, $name ]; - $recips{$email} = 1; } - return () unless $all_confirmed; - return keys %recips; + return $all_confirmed && @{$self->to}; } 1; diff --git a/perllib/FixMyStreet/SendReport/Zurich.pm b/perllib/FixMyStreet/SendReport/Zurich.pm index d46561e9e..40417b41e 100644 --- a/perllib/FixMyStreet/SendReport/Zurich.pm +++ b/perllib/FixMyStreet/SendReport/Zurich.pm @@ -26,7 +26,7 @@ sub build_recipient_list { } push @{ $self->to }, [ $body_email, $body->name ]; - return $body_email; + return 1; } sub get_template { diff --git a/t/app/model/problem.t b/t/app/model/problem.t index 6ad933db8..c40e9e022 100644 --- a/t/app/model/problem.t +++ b/t/app/model/problem.t @@ -5,9 +5,9 @@ use warnings; use Test::More; +use FixMyStreet::TestMech; use FixMyStreet; use FixMyStreet::App; -use FixMyStreet::TestMech; use mySociety::Locale; use Sub::Override; diff --git a/t/app/sendreport/email.t b/t/app/sendreport/email.t index 8063c928f..b2cab42ed 100644 --- a/t/app/sendreport/email.t +++ b/t/app/sendreport/email.t @@ -50,14 +50,14 @@ foreach my $test ( { }, { desc => 'unconfirmed contact results in no receipients', - count => undef, + count => 0, add_council => 1, unconfirmed => 1, expected_note => 'Body 1000 deleted', }, { desc => 'unconfirmed contact note uses note from contact table', - count => undef, + count => 0, add_council => 1, unconfirmed => 1, note => 'received bounced so unconfirmed', -- cgit v1.2.3 From e0f5f22f770c146986fe5bbdd083fbb48eb9706d Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Tue, 13 Jan 2015 17:15:23 +0000 Subject: Add some bottom padding to .content. And remove it in footer if present (so that a different background doesn't potentially show through). --- web/cobrands/sass/_layout.scss | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/web/cobrands/sass/_layout.scss b/web/cobrands/sass/_layout.scss index 632385cac..607734c41 100644 --- a/web/cobrands/sass/_layout.scss +++ b/web/cobrands/sass/_layout.scss @@ -388,10 +388,11 @@ body.fullwidthpage { .container { .content { box-shadow: none; - padding: 1em 1em 0em; + padding: 1em; margin-bottom: 0em; footer { margin-top: 0em; + margin-bottom: -1em; .tablewrapper { padding-top: 2em; padding-bottom: 3em; -- cgit v1.2.3 From 6bbca382a56d4a31f60da5de7ba689c15c1222ef Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Wed, 14 Jan 2015 16:30:33 +0000 Subject: Update other uses of text changed in a0a33741. --- templates/web/barnet/report/new/councils_text_all.html | 2 +- templates/web/base/report/new/councils_extra_text.html | 2 +- templates/web/base/report/new/councils_text_some.html | 5 ++--- templates/web/base/report/new/fill_in_details_form.html | 2 +- templates/web/oxfordshire/report/new/councils_text_all.html | 4 ++-- 5 files changed, 7 insertions(+), 8 deletions(-) diff --git a/templates/web/barnet/report/new/councils_text_all.html b/templates/web/barnet/report/new/councils_text_all.html index 20152928b..ae02f7647 100644 --- a/templates/web/barnet/report/new/councils_text_all.html +++ b/templates/web/barnet/report/new/councils_text_all.html @@ -1,4 +1,4 @@

All the information you provide here will be sent to the relevant department. -[% loc('The subject and details of the problem will be public, plus your name if you give us permission.') %] +[% loc('The summary and description will also be made public (see our privacy policy).') %]

diff --git a/templates/web/base/report/new/councils_extra_text.html b/templates/web/base/report/new/councils_extra_text.html index 1088b9f8b..b77758b01 100644 --- a/templates/web/base/report/new/councils_extra_text.html +++ b/templates/web/base/report/new/councils_extra_text.html @@ -1 +1 @@ -[% loc('The subject and details of the problem will be public, plus your name if you give us permission.') %] +[% loc('The summary and description will also be made public (see our privacy policy).') %] diff --git a/templates/web/base/report/new/councils_text_some.html b/templates/web/base/report/new/councils_text_some.html index 8d4768612..70cb97617 100644 --- a/templates/web/base/report/new/councils_text_some.html +++ b/templates/web/base/report/new/councils_text_some.html @@ -7,9 +7,8 @@ [%- '.' IF loop.last %] [% END %] -[% - loc('The subject and details of the problem will be public, plus your name if you give us permission.'); -%] +[% loc('The summary and description will also be made public (see our privacy policy).') %] + [% nget( "We do not yet have details for the other council that covers this location.", diff --git a/templates/web/base/report/new/fill_in_details_form.html b/templates/web/base/report/new/fill_in_details_form.html index ec05e127e..cee0244c7 100644 --- a/templates/web/base/report/new/fill_in_details_form.html +++ b/templates/web/base/report/new/fill_in_details_form.html @@ -15,7 +15,7 @@ loc('the local council') ); %] - [% loc('The subject and details of the problem will be public, plus your name if you give us permission.') %] + [% loc('The summary and description will also be made public (see our privacy policy).') %]

[% ELSE %] [% PROCESS 'report/new/councils_text.html' %] diff --git a/templates/web/oxfordshire/report/new/councils_text_all.html b/templates/web/oxfordshire/report/new/councils_text_all.html index f205418d7..a8c81f39e 100644 --- a/templates/web/oxfordshire/report/new/councils_text_all.html +++ b/templates/web/oxfordshire/report/new/councils_text_all.html @@ -1,5 +1,5 @@

All the information you provide here will be sent to the - relevant department. The subject and details of the - problem will be public, plus your name if you give us permission. + relevant department. The summary and description + will also be made public, plus your name if you give us permission.

-- cgit v1.2.3 From 78bf9e3589f14436befb73e903ec574d273bf66f Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Wed, 14 Jan 2015 16:32:38 +0000 Subject: Fix title of Privacy page, and missing copy. --- .../fixmystreet-uk-councils/static/privacy.html | 131 +++++++++++++++++++++ templates/web/fixmystreet.com/static/privacy.html | 2 +- 2 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 templates/web/fixmystreet-uk-councils/static/privacy.html diff --git a/templates/web/fixmystreet-uk-councils/static/privacy.html b/templates/web/fixmystreet-uk-councils/static/privacy.html new file mode 100644 index 000000000..addce1e2d --- /dev/null +++ b/templates/web/fixmystreet-uk-councils/static/privacy.html @@ -0,0 +1,131 @@ +[% INCLUDE 'header.html', + title = loc('Privacy and cookies'), + bodyclass = 'twothirdswidthpage' %] + +
+ +
+ +

Privacy, cookies, and third party services

+ +

Our use of your data, cookies, and external services: what you +should know, and how to opt out if you want to.

+ +

Summary: We care a lot about our users’ privacy. We provide details below, +and we try our hardest to look after the private data that we hold. Like many +other websites, we sometimes use cookies and Google Analytics to help us make +our websites better. These tools are very common and used by many other sites, +but they do have privacy implications, and as a charity concerned with socially +positive uses of the internet, we think it’s important to explain them in full. +If you don’t want to share your browsing activities on mySociety’s sites with +other companies, you can adjust your usage or install opt-out browser plugins. + +

Privacy

+ +
+
Who gets to see my email address?
+
If you submit a problem, we pass on your details, and details of +the problem, to the council contact or contacts responsible for the area where +you located the problem, or other relevant body (for example, in most of London +we send reports via the central London Report-It service, who may also publish +the public details of your report). Other than the council, who obviously get +your email address, only people we authorise to view the FixMyStreet +administration interface will be able to see your email address and they will +never use it for anything other than to help administer FixMyStreet. Similarly +with email addresses from updates. We will never give or sell your email +address to anyone else, unless we are obliged to by law. Your name, given in +the name field, will not be published anywhere unless you let us (if you give +your name elsewhere, e.g. in the public details section, it will be publicly +available).
+
Will you send nasty, brutish spam to my email address?
+
Never. We will email you if someone leaves an update on a +problem you’ve reported, and send you a questionnaire email four weeks +after you submit a problem, asking for a status update; we’ll only ever +send you emails in relation to your problem.
+
+ +

Cookies

+ +

To make our service easier or more useful, we sometimes place small data +files on your computer or mobile phone, known as cookies; many websites do +this. We use this information to, for example, remember you have logged in so +you don't need to do that on every page, or to measure how people use the +website so we can improve it and make sure it works properly. Below, we list +the cookies and services that this site can use. + + + + + + + + + + + + + + +
NameTypical ContentExpires
fixmystreet_app_sessionA random unique identifierWhen browser is closed, or four weeks if “Keep me signed in” is ticked
+ +

Measuring website usage (Google Analytics)

+ +

We use Google Analytics to collect information about how people use this +site. We do this to make sure it’s meeting its users’ needs and to understand +how we could do it better. Google Analytics stores information such as what +pages you visit, how long you are on the site, how you got here, what you click +on, and information about your web browser. IP addresses are masked (only a +portion is stored) and personal information is only reported in aggregate. We +do not allow Google to use or share our analytics data for any purpose besides +providing us with analytics information, and we recommend that any user of +Google Analytics does the same. + +

If you’re unhappy with data about your visit to be used in this way, you can +install the official browser +plugin for blocking Google Analytics. + +

The cookies set by Google Analytics are as follows: + + + + + + + +
NameTypical ContentExpires
__utmaUnique anonymous visitor ID2 years
__utmbUnique anonymous session ID30 minutes
__utmzInformation on how the site was reached (e.g. direct or via a link/search/advertisement)6 months
__utmxWhich variation of a page you are seeing if we are testing different versions to see which is best2 years
+ +

Google’s Official Statement about Analytics Data

+ +

“This website uses Google Analytics, a web analytics service provided by +Google, Inc. (“Google”). Google Analytics uses “cookies”, which are text files +placed on your computer, to help the website analyze how users use the site. +The information generated by the cookie about your use of the website +(including your IP address) will be transmitted to and stored by Google on +servers in the United States . Google will use this information for the purpose +of evaluating your use of the website, compiling reports on website activity +for website operators and providing other services relating to website activity +and internet usage. Google may also transfer this information to third parties +where required to do so by law, or where such third parties process the +information on Google’s behalf. Google will not associate your IP address with +any other data held by Google. You may refuse the use of cookies by selecting +the appropriate settings on your browser, however please note that if you do +this you may not be able to use the full functionality of this website. By +using this website, you consent to the processing of data about you by Google +in the manner and for the purposes set out above.”

+ +

More general information +on how third party services work

+ +

Credits

+ +

Bits of wording taken from the gov.uk cookies page (under the Open +Government Licence). + +[% INCLUDE 'footer.html' pagefooter = 'yes' %] diff --git a/templates/web/fixmystreet.com/static/privacy.html b/templates/web/fixmystreet.com/static/privacy.html index e7c78384f..addce1e2d 100755 --- a/templates/web/fixmystreet.com/static/privacy.html +++ b/templates/web/fixmystreet.com/static/privacy.html @@ -1,5 +1,5 @@ [% INCLUDE 'header.html', - title = loc('Frequently Asked Questions'), + title = loc('Privacy and cookies'), bodyclass = 'twothirdswidthpage' %]

-- cgit v1.2.3 From 7fa239a9c2122074bb65bbb0ac7d30d922a4f761 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Tue, 13 Jan 2015 17:36:15 +0000 Subject: Use public bundles for setting up cache. Forked repos cannot access the secure credentials when making pull requests. We'd like those to be tested, so have made the bundles public to be used for installation. --- .travis/install | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/.travis/install b/.travis/install index 8514e801d..c766d3598 100755 --- a/.travis/install +++ b/.travis/install @@ -4,28 +4,22 @@ import hashlib import os import sys import tarfile - -import boto -from boto.s3.key import Key -from boto.exception import S3ResponseError +import urllib root = os.path.join(os.path.dirname(__file__), '..') with open(os.path.join(root, 'cpanfile.snapshot')) as cpanfile: hash = hashlib.md5(cpanfile.read()).hexdigest() wanted_filename = 'fixmystreet-local-%s.tgz' % hash -conn = boto.connect_s3() -bucket = conn.get_bucket('fixmystreet-bundle-cache') - -k = Key(bucket) -k.key = wanted_filename +url = 'https://fixmystreet-bundle-cache.s3.amazonaws.com/%s' % wanted_filename try: - k.get_contents_to_filename(wanted_filename) + urllib.urlretrieve(url, wanted_filename) if tarfile.is_tarfile(wanted_filename): tfile = tarfile.open(wanted_filename) tfile.extractall() + print "Cached copy found and extracted" sys.exit(0) -except S3ResponseError: +except IOError: os.remove(wanted_filename) print "No cached copy found, running carton install..." -- cgit v1.2.3 From 21c8fded9bfd34c456f5202bc812408152f3b6cf Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Fri, 16 Jan 2015 17:20:30 +0000 Subject: Fix tiny display bug in map sub links navigation. --- web/cobrands/sass/_layout.scss | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/web/cobrands/sass/_layout.scss b/web/cobrands/sass/_layout.scss index 607734c41..d1cfb6be7 100644 --- a/web/cobrands/sass/_layout.scss +++ b/web/cobrands/sass/_layout.scss @@ -631,8 +631,8 @@ body.twothirdswidthpage { display:block; cursor: pointer; position:absolute; - left:-1em; - width:1em; + left: -0.97em; /* 1em leaves a tiny gap, font issue */ + width: 1em; height:100%; background:#000 url($image-sprite) right -4119px no-repeat; @include border-radius(0.25em 0 0 0.25em); -- cgit v1.2.3 From 2e8df1a5a6610c43e0c1bda15d018fa16738061b Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Mon, 19 Jan 2015 16:20:55 +0000 Subject: Tidy up of bin directory. Remove some unneeded scripts, move others to cobrand-specific directories. --- .travis.yml | 2 +- bin/canonicalise-csv | 112 ------------ bin/canonicalise-eha | 80 --------- bin/comment-backfill | 21 --- bin/emptyhomes/canonicalise-eha | 80 +++++++++ bin/emptyhomes/make_welsh_po | 114 +++++++++++++ bin/export-norwegian-contacts | 97 ----------- bin/fiksgatami/export-norwegian-contacts | 97 +++++++++++ bin/fiksgatami/load-norwegian-contacts | 69 ++++++++ bin/fixmystreet.com/canonicalise-csv | 112 ++++++++++++ bin/fixmystreet.com/generate_council_location | 28 +++ bin/fixmystreet.com/import-categories | 78 +++++++++ bin/fixmystreet.com/load-contacts | 49 ++++++ bin/fixmystreet.com/populate_bing_cache | 60 +++++++ bin/fixmystreet.com/rotate-photos | 64 +++++++ bin/fixmystreet.com/showcouncilrates | 74 ++++++++ bin/fixmystreet.com/update-areas | 46 +++++ bin/fixmystreet.com/update-send-questionnaire | 29 ++++ bin/generate_council_location | 28 --- bin/import-categories | 78 --------- bin/install-as-user | 2 +- bin/kasabi | 234 -------------------------- bin/load-contacts | 49 ------ bin/load-norwegian-contacts | 69 -------- bin/make_emptyhomes_welsh_po | 114 ------------- bin/populate_bing_cache | 60 ------- bin/rotate-photos | 64 ------- bin/showcouncilrates | 74 -------- bin/update-areas | 46 ----- bin/update-send-questionnaire | 29 ---- bin/zerotb/import_clinic_list.pl | 115 +++++++++++++ bin/zerotb_import_clinic_list.pl | 115 ------------- bin/zurich-overdue-alert | 83 --------- bin/zurich/geocode | 45 ----- bin/zurich/overdue-alert | 83 +++++++++ data/kasabi-requirements.txt | 2 - perllib/FixMyStreet/DB/Result/Token.pm | 5 - perllib/Utils.pm | 1 - 38 files changed, 1100 insertions(+), 1408 deletions(-) delete mode 100755 bin/canonicalise-csv delete mode 100755 bin/canonicalise-eha delete mode 100755 bin/comment-backfill create mode 100755 bin/emptyhomes/canonicalise-eha create mode 100755 bin/emptyhomes/make_welsh_po delete mode 100644 bin/export-norwegian-contacts create mode 100644 bin/fiksgatami/export-norwegian-contacts create mode 100755 bin/fiksgatami/load-norwegian-contacts create mode 100755 bin/fixmystreet.com/canonicalise-csv create mode 100755 bin/fixmystreet.com/generate_council_location create mode 100755 bin/fixmystreet.com/import-categories create mode 100755 bin/fixmystreet.com/load-contacts create mode 100755 bin/fixmystreet.com/populate_bing_cache create mode 100755 bin/fixmystreet.com/rotate-photos create mode 100755 bin/fixmystreet.com/showcouncilrates create mode 100755 bin/fixmystreet.com/update-areas create mode 100755 bin/fixmystreet.com/update-send-questionnaire delete mode 100755 bin/generate_council_location delete mode 100755 bin/import-categories delete mode 100755 bin/kasabi delete mode 100755 bin/load-contacts delete mode 100755 bin/load-norwegian-contacts delete mode 100755 bin/make_emptyhomes_welsh_po delete mode 100755 bin/populate_bing_cache delete mode 100755 bin/rotate-photos delete mode 100755 bin/showcouncilrates delete mode 100755 bin/update-areas delete mode 100755 bin/update-send-questionnaire create mode 100755 bin/zerotb/import_clinic_list.pl delete mode 100755 bin/zerotb_import_clinic_list.pl delete mode 100755 bin/zurich-overdue-alert delete mode 100755 bin/zurich/geocode create mode 100755 bin/zurich/overdue-alert delete mode 100644 data/kasabi-requirements.txt diff --git a/.travis.yml b/.travis.yml index bad676933..1e4b7611f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,7 +33,7 @@ before_script: sed -r -e "s,(FMS_DB_USER:) 'fms',\\1 'postgres'," conf/general.yml-example > conf/general.yml - ./bin/cron-wrapper ./bin/make_po FixMyStreet-EmptyHomes - - ./bin/cron-wrapper ./bin/make_emptyhomes_welsh_po + - ./bin/cron-wrapper ./bin/emptyhomes/make_welsh_po - commonlib/bin/gettext-makemo FixMyStreet script: "bin/cron-wrapper perl /usr/bin/prove -rl t" after_script: diff --git a/bin/canonicalise-csv b/bin/canonicalise-csv deleted file mode 100755 index c0a7fc60b..000000000 --- a/bin/canonicalise-csv +++ /dev/null @@ -1,112 +0,0 @@ -#!/usr/bin/perl -w - -# canonicalise-csv: -# Convert provided CSV file into one with standard names for MaPit -# -# Copyright (c) 2006 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: canonicalise-csv,v 1.4 2009-02-16 18:56:44 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::MaPit; -use mySociety::VotingArea; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::MaPit::configure(); -} - -my %councils; -open(FP, "$FindBin::Bin/../data/councils.csv") - or die "Can't read councils.csv: $!\n"; -while () { - s/\r?\n//g; - my ($name, $email) = split /,/; - $email ||= ''; - $email =~ s/\xa0//g; - - # Canonicalise - next if $name eq 'Londonderry'; # Dupe of Derry - next if $name eq 'Kingston upon Hull'; # Dupe of Hull - next if $name eq 'London' || $name eq 'Greater London'; # Untrustworthy - - # Different - $name = 'Corporation of London' if $name eq 'City of London'; - $name = "Renfrewsh'r" if $name eq 'Renfrewshire'; - # Shorter - $name = 'Blackburn' if $name eq 'Blackburn with Darwen'; - $name = 'Dungannon' if $name eq 'Dungannon & South Tyrone'; - $name = 'Staffordshire' if $name eq 'Staffordshire County'; - $name = 'Armagh' if $name eq 'Armagh City'; - # Longer - $name = 'Kingston upon Hull' if $name eq 'Hull'; - $name = "City of $name" if $name =~ /^(Edinburgh|Glasgow|York)$/; - $name .= ' Islands' if $name eq 'Shetland'; - $name .= ' & Chelsea' if $name eq 'Kensington'; - # Wrong - $name =~ s/King's Lynn/Kings Lynn/; - $name = 'Surrey Heath' if $name eq 'Surrey Health'; - $name = 'Barking & Dagenham' if $name eq 'Barking-Dagenham'; - $name = 'Newtownabbey' if $name eq 'Newtonabbey'; - $name = 'Isles of Scilly' if $name eq 'Isle of Scilly'; - # Compass - $name =~ s/North East /N. E. /; - $name =~ s/^North West /N. W. /; - $name =~ s/^North (?!Lincolnshire|Norfolk|Somerset)/N. /; - $name =~ s/^South (?!Shropshire|Staffordshire|Somerset)/S. /; - $name =~ s/^East (?!Staffordshire)/E. /; - $name =~ s/^West(ern)? (?!Berkshire|Wiltshire)/W. /; - $name =~ s/ W / W. /; - # Various - $name =~ s/^Great /Gt. /; - $name =~ s/^St /St. /; - $name =~ s/ and / & /; - $name =~ s/ ?Royal$//; - $name =~ s/ Borough$//; - $name =~ s/-(upon|on|le)-/ $1 /; - $councils{$name} = $email; -} -close(FP); - -my $types = $mySociety::VotingArea::council_parent_types; -my (%out, @missing); -foreach my $type (@$types) { - my $areas = mySociety::MaPit::get_areas_by_type($type); - my $areas_info = mySociety::MaPit::get_voting_areas_info($areas); - foreach my $id (keys %$areas_info) { - my $area_info = $areas_info->{$id}; - my $name = $area_info->{name}; - if ($name eq 'Durham City Council') { - $out{$id} = $councils{'Durham City'}; - next; - } elsif ($name eq 'Durham County Council') { - $out{$id} = $councils{'Durham County'}; - next; - } - $name =~ s/( (Borough|City|District|County))* Council//; - if ($councils{$name} && $councils{$name} =~ /@/) { - $out{$id} = $councils{$name}; - } elsif ($councils{$name} || exists($councils{$name})) { - push @missing, $id; - } - } -} - -# Output missing IDs to STDOUT -print join(',', @missing) . "\n"; - -# Output emails to canonical CSV -open(FP, ">$FindBin::Bin/../data/councils_canonical.csv"); -foreach (sort keys %out) { - print FP "$_," . $out{$_} . "\n"; -} -close FP; diff --git a/bin/canonicalise-eha b/bin/canonicalise-eha deleted file mode 100755 index 1030982fa..000000000 --- a/bin/canonicalise-eha +++ /dev/null @@ -1,80 +0,0 @@ -#!/usr/bin/perl -w - -# canonicalise-eha: -# Convert provided TSV file into one with standard names for MaPit -# -# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: canonicalise-eha,v 1.1 2009-05-27 11:11:18 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::MaPit; -use mySociety::VotingArea; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::MaPit::configure(); -} - -my %councils; -open(FP, "$FindBin::Bin/../data/eha_listN"); -while () { - s/\r?\n//g; - my ($name, $email) = split /\s*\t+\s*/; - $email ||= ''; - #$name = 'Blackburn' if $name eq 'Blackburn with Darwen'; - $name =~ s/^St /St. /; - $name =~ s/-(upon|on|le)-/ $1 /; - $name =~ s/^LB //; - $name =~ s/ BC| DC| MDC| MBC| (Metropolitan|Borough|District|County|City) Council| Council//; - print "Already have $name\n" if $councils{$name}; - $councils{$name} = $email; -} -close(FP); - -my @types = grep { $_ !~ /CTY|LGD/ } @$mySociety::VotingArea::council_parent_types; -my (%out); -foreach my $type (@types) { - my $areas = mySociety::MaPit::get_areas_by_type($type); - my $areas_info = mySociety::MaPit::get_voting_areas_info($areas); - foreach my $id (keys %$areas_info) { - my $area_info = $areas_info->{$id}; - next if $area_info->{country} eq 'S'; - my $name = $area_info->{name}; - if ($name eq 'Durham City Council') { - $out{$id} = $councils{'Durham City'}; - delete $councils{'Durham City'}; - next; - #} elsif ($name eq 'Durham County Council') { - # $out{$id} = $councils{'Durham County'}; - # next; - } - $name =~ s/( (Borough|City|District|County))* Council//; - if ($councils{$name} && $councils{$name} =~ /@/) { - $out{$id} = $councils{$name}; - delete $councils{$name}; - } else { - print "Problem: $name $area_info->{country}\n"; - } - } -} - -foreach (sort keys %councils) { - print "Not matched: $_ $councils{$_}\n"; -} - -# Output emails to canonical CSV -open(FP, ">$FindBin::Bin/../data/eha.csv"); -foreach (sort keys %out) { - print FP "$_," . $out{$_} . "\n"; -} -close FP; diff --git a/bin/comment-backfill b/bin/comment-backfill deleted file mode 100755 index e296d7756..000000000 --- a/bin/comment-backfill +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; -require 5.8.0; -use DateTime; - -use FixMyStreet::App; - -use Open311; -use Open311::GetServiceRequestUpdates; - -my $start_time = DateTime->now->subtract( days => 1, hours => 1 ); -my $end_time = DateTime->now; - -my $updates = Open311::GetServiceRequestUpdates->new( - start_date => $start_time, - end_date => $end_time, -); - -$updates->fetch; diff --git a/bin/emptyhomes/canonicalise-eha b/bin/emptyhomes/canonicalise-eha new file mode 100755 index 000000000..1030982fa --- /dev/null +++ b/bin/emptyhomes/canonicalise-eha @@ -0,0 +1,80 @@ +#!/usr/bin/perl -w + +# canonicalise-eha: +# Convert provided TSV file into one with standard names for MaPit +# +# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: canonicalise-eha,v 1.1 2009-05-27 11:11:18 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::MaPit; +use mySociety::VotingArea; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::MaPit::configure(); +} + +my %councils; +open(FP, "$FindBin::Bin/../data/eha_listN"); +while () { + s/\r?\n//g; + my ($name, $email) = split /\s*\t+\s*/; + $email ||= ''; + #$name = 'Blackburn' if $name eq 'Blackburn with Darwen'; + $name =~ s/^St /St. /; + $name =~ s/-(upon|on|le)-/ $1 /; + $name =~ s/^LB //; + $name =~ s/ BC| DC| MDC| MBC| (Metropolitan|Borough|District|County|City) Council| Council//; + print "Already have $name\n" if $councils{$name}; + $councils{$name} = $email; +} +close(FP); + +my @types = grep { $_ !~ /CTY|LGD/ } @$mySociety::VotingArea::council_parent_types; +my (%out); +foreach my $type (@types) { + my $areas = mySociety::MaPit::get_areas_by_type($type); + my $areas_info = mySociety::MaPit::get_voting_areas_info($areas); + foreach my $id (keys %$areas_info) { + my $area_info = $areas_info->{$id}; + next if $area_info->{country} eq 'S'; + my $name = $area_info->{name}; + if ($name eq 'Durham City Council') { + $out{$id} = $councils{'Durham City'}; + delete $councils{'Durham City'}; + next; + #} elsif ($name eq 'Durham County Council') { + # $out{$id} = $councils{'Durham County'}; + # next; + } + $name =~ s/( (Borough|City|District|County))* Council//; + if ($councils{$name} && $councils{$name} =~ /@/) { + $out{$id} = $councils{$name}; + delete $councils{$name}; + } else { + print "Problem: $name $area_info->{country}\n"; + } + } +} + +foreach (sort keys %councils) { + print "Not matched: $_ $councils{$_}\n"; +} + +# Output emails to canonical CSV +open(FP, ">$FindBin::Bin/../data/eha.csv"); +foreach (sort keys %out) { + print FP "$_," . $out{$_} . "\n"; +} +close FP; diff --git a/bin/emptyhomes/make_welsh_po b/bin/emptyhomes/make_welsh_po new file mode 100755 index 000000000..a85e7c35b --- /dev/null +++ b/bin/emptyhomes/make_welsh_po @@ -0,0 +1,114 @@ +#!/usr/bin/perl -w +use strict; + +# Given a translation of the Empty Homes version of the FixMyStreet .po file, +# fix it all up to go direct from FixMyStreet -> EH translation. + +use POSIX; +use FindBin; +use lib "$FindBin::Bin/../../perllib"; +use PoChange; + +chdir("$FindBin::Bin/../../locale"); + +# First read in translation and match up. +open(INPO, 'cy_GB.UTF-8/LC_MESSAGES/EmptyHomes.po') or die $!; + +my $state = 'start'; +my $msgid = ''; +my $fuzzy = 0; +my $msgstr; +my %lookup; +while () { + if (m/^#, fuzzy/) { + $fuzzy = 1; + } elsif (m/^#/) { + # comment or blank line + } elsif (m/^\s+$/) { + # blank line separates translations + $msgid =~ s/"\n"//g; + $lookup{$msgid} = [ $msgstr, $fuzzy ]; + $state = 'msgid'; + $msgid = ""; + $fuzzy = 0; + } elsif ($state eq 'msgid' && (m/^msgstr "/ || m/^msgstr\[0\] "/)) { + $msgstr = $_; + $state = 'msgstr'; + } elsif ($state eq 'msgstr') { + $msgstr .= $_; + } elsif ($state eq 'msgid') { + $msgid .= $_; + } +} +$lookup{$msgid} = [ $msgstr, $fuzzy ]; +close INPO; + +mkdir("en_GB.UTF-8"); +mkdir("en_GB.UTF-8/LC_MESSAGES"); +mkdir("cy_GB.UTF-8"); +mkdir("cy_GB.UTF-8/LC_MESSAGES"); + +open(MAINPO, 'FixMyStreet.po') or die; +open(OUTPO, ">cy_GB.UTF-8/LC_MESSAGES/FixMyStreet-EmptyHomes.po") or die; + +print OUTPO "# AUTOMATICALLY GENERATED by make_welsh_po, do not edit\n\n"; + +my $buffer = ""; +my $start = 0; +while() { + if (!$start) { + s/#, fuzzy/#/; + } + if (m/"Last-Translator: FULL NAME/) { + $_ = '"Last-Translator: mysociety/bin/emptyhomes/make_welsh_po\\n"'."\n"; + } + if (m/"PO-Revision-Date: YEAR-MO-DA/) { + my $time = POSIX::strftime("%Y-%m-%d %H:%M%z", localtime(time())); + $_ = '"PO-Revision-Date: '.$time.'\\n"'."\n"; + } + if (m/"Language-Team: LANGUAGE/) { + $_ = '"Language-Team: mySociety\\n"'."\n"; + } + if (m/"Plural-Forms: nplurals=/) { + $_ = '"Plural-Forms: nplurals=2; plural=n != 1;\\n"'."\n"; + } + + if (m/^#/) { + # comment or blank line + print OUTPO $_; + } elsif (m/^\s+$/) { + # blank line + $start = 1; + $buffer = ""; + print OUTPO $_; + } elsif ($start && (m/^msgstr "/ || m/^msgstr\[0\] "/)) { + # start of translated text - translate English into Empty Homes language + my $new_buffer = PoChange::fixmystreet_to_reportemptyhomes($buffer); + $new_buffer =~ s/"\n"//g; + + if ($lookup{$new_buffer} && $lookup{$new_buffer}[1]) { + print OUTPO "#, fuzzy\n"; + } + + print OUTPO $buffer; + if ($lookup{$new_buffer}) { + print OUTPO $lookup{$new_buffer}[0]; + } else { + if (m/^msgstr\[0\] ""/) { + $new_buffer =~ s/^msgid "/msgstr[0] "/m; + $new_buffer =~ s/^msgid_plural "/msgstr[1] "/m; + $_ = ; # skip untranslated plural + } else { + $new_buffer =~ s/^msgid "/msgstr "/; + } + print OUTPO $new_buffer; + } + + $buffer = ""; + } elsif (!$start) { + print OUTPO $_; + } else { + $buffer .= $_; + } +} + diff --git a/bin/export-norwegian-contacts b/bin/export-norwegian-contacts deleted file mode 100644 index 0209c6512..000000000 --- a/bin/export-norwegian-contacts +++ /dev/null @@ -1,97 +0,0 @@ -#!/usr/bin/perl - -# export-norwegian-contacts: -# Export initial contact list from fiksgatami in a format usable by -# load-norwegian-contact. -# -# The format is -# ID;Name;email-address;Category1,Category2,... -# -# Based on script load-contacts copyright (c) 2006 UK Citizens Online -# Democracy and script load-norwegian-contacts copyright (c) 2011 -# Petter Reinholdtsen. -# Copyright 2011 Petter Reinholdtsen. -# -# $Id: load-norwegian-contacts,v 1.0 2007-08-02 11:44:59 matthew Exp $ - -use strict; -use warnings; -require 5.8.0; -use open OUT => ':utf8'; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::MaPit; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); -} - -my $datafile = shift; - -open(my $fh, '>', $datafile) or die "Unable to write to $datafile"; - -# Categories used by more than half the areas is used as the default -# list. -my $sql = - "SELECT category FROM (SELECT category, COUNT(*) from contacts ". - " WHERE confirmed = 'true' AND deleted = 'false' ". - " GROUP BY category) as c ". - " WHERE count > (SELECT COUNT(*)/2 FROM contacts ". - " WHERE category = 'Annet') ". - " ORDER BY category"; -my $defcategoriesref = dbh()->selectcol_arrayref($sql); -print $fh "0000;default;;", join(',', @{$defcategoriesref}), "\n"; - -my %categorygroup; -$sql = "SELECT area_id, email, category FROM contacts ORDER BY category"; -my $contactref = dbh()->selectall_arrayref($sql, { Slice => {} }); -my @area_ids; -for my $row (@{$contactref}) { - my $key = $row->{area_id} .':'. $row->{email}; - push(@area_ids, $row->{area_id}); - if (exists $categorygroup{$key}) { - push(@{$categorygroup{$key}}, $row->{category}); - } else { - $categorygroup{$key} = [ $row->{category} ]; - } -} - -my $areas_info = mySociety::MaPit::call('areas', \@area_ids); - -my @list; -for my $key (keys %categorygroup) { - my ($area_id, $email) = split(/:/, $key); - my $categoriesref = $categorygroup{$key}; - my $areaname = $areas_info->{$area_id}->{name}; - my $areadigits = ($area_id < 100) ? 2 : 4; - if (identical_lists($defcategoriesref, $categoriesref)) { - push(@list,sprintf("%0${areadigits}d;%s;%s\n", - $area_id, $areaname, $email)); - } else { - push(@list, sprintf("%0${areadigits}d;%s;%s;%s\n", - $area_id, $areaname, $email, - join(',', @{$categoriesref}))); - } -} - -print $fh sort @list; - -sub identical_lists { - my ($a, $b) = @_; - return !(join(',', @{$a}) cmp join(',', @{$b})); -} - -exit 0; diff --git a/bin/fiksgatami/export-norwegian-contacts b/bin/fiksgatami/export-norwegian-contacts new file mode 100644 index 000000000..0209c6512 --- /dev/null +++ b/bin/fiksgatami/export-norwegian-contacts @@ -0,0 +1,97 @@ +#!/usr/bin/perl + +# export-norwegian-contacts: +# Export initial contact list from fiksgatami in a format usable by +# load-norwegian-contact. +# +# The format is +# ID;Name;email-address;Category1,Category2,... +# +# Based on script load-contacts copyright (c) 2006 UK Citizens Online +# Democracy and script load-norwegian-contacts copyright (c) 2011 +# Petter Reinholdtsen. +# Copyright 2011 Petter Reinholdtsen. +# +# $Id: load-norwegian-contacts,v 1.0 2007-08-02 11:44:59 matthew Exp $ + +use strict; +use warnings; +require 5.8.0; +use open OUT => ':utf8'; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::MaPit; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); +} + +my $datafile = shift; + +open(my $fh, '>', $datafile) or die "Unable to write to $datafile"; + +# Categories used by more than half the areas is used as the default +# list. +my $sql = + "SELECT category FROM (SELECT category, COUNT(*) from contacts ". + " WHERE confirmed = 'true' AND deleted = 'false' ". + " GROUP BY category) as c ". + " WHERE count > (SELECT COUNT(*)/2 FROM contacts ". + " WHERE category = 'Annet') ". + " ORDER BY category"; +my $defcategoriesref = dbh()->selectcol_arrayref($sql); +print $fh "0000;default;;", join(',', @{$defcategoriesref}), "\n"; + +my %categorygroup; +$sql = "SELECT area_id, email, category FROM contacts ORDER BY category"; +my $contactref = dbh()->selectall_arrayref($sql, { Slice => {} }); +my @area_ids; +for my $row (@{$contactref}) { + my $key = $row->{area_id} .':'. $row->{email}; + push(@area_ids, $row->{area_id}); + if (exists $categorygroup{$key}) { + push(@{$categorygroup{$key}}, $row->{category}); + } else { + $categorygroup{$key} = [ $row->{category} ]; + } +} + +my $areas_info = mySociety::MaPit::call('areas', \@area_ids); + +my @list; +for my $key (keys %categorygroup) { + my ($area_id, $email) = split(/:/, $key); + my $categoriesref = $categorygroup{$key}; + my $areaname = $areas_info->{$area_id}->{name}; + my $areadigits = ($area_id < 100) ? 2 : 4; + if (identical_lists($defcategoriesref, $categoriesref)) { + push(@list,sprintf("%0${areadigits}d;%s;%s\n", + $area_id, $areaname, $email)); + } else { + push(@list, sprintf("%0${areadigits}d;%s;%s;%s\n", + $area_id, $areaname, $email, + join(',', @{$categoriesref}))); + } +} + +print $fh sort @list; + +sub identical_lists { + my ($a, $b) = @_; + return !(join(',', @{$a}) cmp join(',', @{$b})); +} + +exit 0; diff --git a/bin/fiksgatami/load-norwegian-contacts b/bin/fiksgatami/load-norwegian-contacts new file mode 100755 index 000000000..b73778848 --- /dev/null +++ b/bin/fiksgatami/load-norwegian-contacts @@ -0,0 +1,69 @@ +#!/usr/bin/perl -w + +# load-norwegian-contacts: +# Load initial contact list into fiksgatami. +# The format is +# Name;ID;email-address;Category1,Category2,... +# +# Based on script load-contacts copyright (c) 2006 UK Citizens Online Democracy. +# Copyright 2011 Petter Reinholdtsen +# +# $Id: load-norwegian-contacts,v 1.0 2007-08-02 11:44:59 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::Random; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); + + if (!dbh()->selectrow_array('select secret from secret for update of secret')) { + local dbh()->{HandleError}; + dbh()->do('insert into secret (secret) values (?)', {}, unpack('h*', mySociety::Random::random_bytes(32))); + } + dbh()->commit(); +} + +my $datafile = shift; +my $defcategories; +open(FP, "<", $datafile) or die "Unable to read from $datafile"; +while () { + s/\r?\n//g; + print "Loading $_\n"; + my ($id, $name, $email, $categories) = split /;/; + if (0 == $id) { + $defcategories = $categories; + next; + } + unless ($categories) { + $categories = $defcategories; + } +# dbh()->do("INSERT INTO contacts (area_id, email, editor, whenedited, note, confirmed, deleted) +# VALUES (?, ?, 'import', ms_current_timestamp(), 'Initial import', 'false', 'false')", +# {}, $id, $email); +# } else { + for my $category (split(/,\s*/, $categories)) { + print " Category '$category'\n"; + dbh()->do("INSERT INTO contacts (area_id, email, category, editor, whenedited, note, confirmed, deleted) + VALUES (?, ?, ?, 'import', ms_current_timestamp(), 'Initial import', 'true', 'false')", + {}, $id, $email, $category); + } +# } +} +dbh()->commit(); +close(FP); diff --git a/bin/fixmystreet.com/canonicalise-csv b/bin/fixmystreet.com/canonicalise-csv new file mode 100755 index 000000000..c0a7fc60b --- /dev/null +++ b/bin/fixmystreet.com/canonicalise-csv @@ -0,0 +1,112 @@ +#!/usr/bin/perl -w + +# canonicalise-csv: +# Convert provided CSV file into one with standard names for MaPit +# +# Copyright (c) 2006 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: canonicalise-csv,v 1.4 2009-02-16 18:56:44 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::MaPit; +use mySociety::VotingArea; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::MaPit::configure(); +} + +my %councils; +open(FP, "$FindBin::Bin/../data/councils.csv") + or die "Can't read councils.csv: $!\n"; +while () { + s/\r?\n//g; + my ($name, $email) = split /,/; + $email ||= ''; + $email =~ s/\xa0//g; + + # Canonicalise + next if $name eq 'Londonderry'; # Dupe of Derry + next if $name eq 'Kingston upon Hull'; # Dupe of Hull + next if $name eq 'London' || $name eq 'Greater London'; # Untrustworthy + + # Different + $name = 'Corporation of London' if $name eq 'City of London'; + $name = "Renfrewsh'r" if $name eq 'Renfrewshire'; + # Shorter + $name = 'Blackburn' if $name eq 'Blackburn with Darwen'; + $name = 'Dungannon' if $name eq 'Dungannon & South Tyrone'; + $name = 'Staffordshire' if $name eq 'Staffordshire County'; + $name = 'Armagh' if $name eq 'Armagh City'; + # Longer + $name = 'Kingston upon Hull' if $name eq 'Hull'; + $name = "City of $name" if $name =~ /^(Edinburgh|Glasgow|York)$/; + $name .= ' Islands' if $name eq 'Shetland'; + $name .= ' & Chelsea' if $name eq 'Kensington'; + # Wrong + $name =~ s/King's Lynn/Kings Lynn/; + $name = 'Surrey Heath' if $name eq 'Surrey Health'; + $name = 'Barking & Dagenham' if $name eq 'Barking-Dagenham'; + $name = 'Newtownabbey' if $name eq 'Newtonabbey'; + $name = 'Isles of Scilly' if $name eq 'Isle of Scilly'; + # Compass + $name =~ s/North East /N. E. /; + $name =~ s/^North West /N. W. /; + $name =~ s/^North (?!Lincolnshire|Norfolk|Somerset)/N. /; + $name =~ s/^South (?!Shropshire|Staffordshire|Somerset)/S. /; + $name =~ s/^East (?!Staffordshire)/E. /; + $name =~ s/^West(ern)? (?!Berkshire|Wiltshire)/W. /; + $name =~ s/ W / W. /; + # Various + $name =~ s/^Great /Gt. /; + $name =~ s/^St /St. /; + $name =~ s/ and / & /; + $name =~ s/ ?Royal$//; + $name =~ s/ Borough$//; + $name =~ s/-(upon|on|le)-/ $1 /; + $councils{$name} = $email; +} +close(FP); + +my $types = $mySociety::VotingArea::council_parent_types; +my (%out, @missing); +foreach my $type (@$types) { + my $areas = mySociety::MaPit::get_areas_by_type($type); + my $areas_info = mySociety::MaPit::get_voting_areas_info($areas); + foreach my $id (keys %$areas_info) { + my $area_info = $areas_info->{$id}; + my $name = $area_info->{name}; + if ($name eq 'Durham City Council') { + $out{$id} = $councils{'Durham City'}; + next; + } elsif ($name eq 'Durham County Council') { + $out{$id} = $councils{'Durham County'}; + next; + } + $name =~ s/( (Borough|City|District|County))* Council//; + if ($councils{$name} && $councils{$name} =~ /@/) { + $out{$id} = $councils{$name}; + } elsif ($councils{$name} || exists($councils{$name})) { + push @missing, $id; + } + } +} + +# Output missing IDs to STDOUT +print join(',', @missing) . "\n"; + +# Output emails to canonical CSV +open(FP, ">$FindBin::Bin/../data/councils_canonical.csv"); +foreach (sort keys %out) { + print FP "$_," . $out{$_} . "\n"; +} +close FP; diff --git a/bin/fixmystreet.com/generate_council_location b/bin/fixmystreet.com/generate_council_location new file mode 100755 index 000000000..c7aea8074 --- /dev/null +++ b/bin/fixmystreet.com/generate_council_location @@ -0,0 +1,28 @@ +#!/usr/bin/perl + +use strict; +use JSON; +use LWP::Simple; + +my $id = shift; + +my $url = sprintf( 'http://mapit.mysociety.org/area/%s/geometry', $id); + +my $json = get( $url ); +my $geo = decode_json( $json ); + +my $max_lat = $geo->{max_lat}; +my $min_lat = $geo->{min_lat}; +my $max_lon = $geo->{max_lon}; +my $min_lon = $geo->{min_lon}; +my $c_lat = $geo->{centre_lat}; +my $c_lon = $geo->{centre_lon}; + +my $spn_lat = $max_lat - $min_lat; +my $spn_lon = $max_lon - $min_lon; + +print <<"EOT"; + centre => '$c_lat,$c_lon', + span => '$spn_lat,$spn_lon', + bounds => [ $min_lat, $min_lon, $max_lat, $max_lon ], +EOT diff --git a/bin/fixmystreet.com/import-categories b/bin/fixmystreet.com/import-categories new file mode 100755 index 000000000..e9008b93f --- /dev/null +++ b/bin/fixmystreet.com/import-categories @@ -0,0 +1,78 @@ +#!/usr/bin/perl -w + +# import-categories: +# Initial choice of categories for councils. Most likely wrong. +# +# Copyright (c) 2007 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: import-categories,v 1.3 2007-08-02 11:44:59 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::MaPit; +use mySociety::VotingArea; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); +} + +my @district_cats = ( + 'Abandoned vehicles', 'Car parking', 'Dog fouling', 'Flyposting', 'Flytipping', + 'Graffiti', 'Parks/landscapes', 'Public toilets', 'Rubbish (refuse and recycling)', + 'Street nameplates', 'Street cleaning', 'Trees' +); +my @county_cats = ( + 'Bus stops', 'Pavements/footpaths', 'Potholes', 'Roads/highways', + 'Road traffic signs', 'Street lighting', 'Traffic lights' +); + +my @councils; +foreach my $type (@$mySociety::VotingArea::council_parent_types) { + my $areas = mySociety::MaPit::get_areas_by_type($type); + push @councils, @$areas; +} +my $councils = mySociety::MaPit::get_voting_areas_info(\@councils); +foreach my $id (keys %$councils) { + my $type = $councils->{$id}->{type}; + my ($email,$confirmed) = dbh()->selectrow_array( + "SELECT email,confirmed FROM contacts WHERE deleted='f' + and area_id=? AND category='Other'", {}, $id); + next unless $email; + if ($type eq 'DIS') { + add_categories($id, $email, $confirmed, @district_cats); + } elsif ($type eq 'CTY') { + add_categories($id, $email, $confirmed, @county_cats); + } else { + add_categories($id, $email, $confirmed, @district_cats, @county_cats); + } +} +dbh()->commit(); + +sub add_categories { + my ($id, $email, $confirmed, @cats) = @_; + foreach (@cats) { + dbh()->do("insert into contacts + (area_id, category, email, editor, whenedited, note, confirmed, deleted) + values + (?, ?, ?, 'import', ms_current_timestamp(), 'Initial copy', ?, 'f')", {}, + $id, $_, $email, ($confirmed ? 1 : 0) + ); + } +} + diff --git a/bin/fixmystreet.com/load-contacts b/bin/fixmystreet.com/load-contacts new file mode 100755 index 000000000..b18699db1 --- /dev/null +++ b/bin/fixmystreet.com/load-contacts @@ -0,0 +1,49 @@ +#!/usr/bin/perl -w + +# canonicalise-csv: +# Convert provided CSV file into one with standard names for MaPit +# +# Copyright (c) 2006 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: load-contacts,v 1.3 2007-08-02 11:44:59 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::Random; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); + + if (!dbh()->selectrow_array('select secret from secret for update of secret')) { + local dbh()->{HandleError}; + dbh()->do('insert into secret (secret) values (?)', {}, unpack('h*', mySociety::Random::random_bytes(32))); + } + dbh()->commit(); +} + +open(FP, "$FindBin::Bin/../data/councils_canonical.csv"); +while () { + s/\r?\n//g; + my ($id, $email) = split /,/; + dbh()->do("INSERT INTO contacts (area_id, email, editor, whenedited, note) + VALUES (?, ?, 'import', ms_current_timestamp(), 'Initial import')", + {}, $id, $email); +} +dbh()->commit(); +close(FP); diff --git a/bin/fixmystreet.com/populate_bing_cache b/bin/fixmystreet.com/populate_bing_cache new file mode 100755 index 000000000..17c8911d0 --- /dev/null +++ b/bin/fixmystreet.com/populate_bing_cache @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +use strict; +use warnings; +require 5.8.0; + +use Data::Dumper; + +use FixMyStreet::App; +use FixMyStreet::Geocode::Bing; + +my $bing_culture = 'en-GB'; + +my $reports = FixMyStreet::App->model('DB::Problem')->search( + { + geocode => undef, + confirmed => { '!=', undef }, + latitude => { '!=', 0 }, + longitude => { '!=', 0 }, + }, + { + select => [qw/id geocode confirmed latitude longitude/], + order_by => { -desc => 'confirmed' } + } +); + +my $num_reports = $reports->count(); +print "Found $num_reports lacking geocode information\n"; + +my $time_to_do = ( $num_reports * 10 ) / 60 / 60; +if ( $time_to_do > 24 ) { + my $days = $time_to_do / 24; + my $hours = $time_to_do % 24; + printf( "Should take %d days and %d hours to finish\n", $days, $hours ); +} +elsif ( $time_to_do < 1 ) { + printf( "Should take %d minutes to finish\n", $time_to_do * 60 ); +} +else { + my $mins = ( $num_reports * 10 ) / 60 % 60; + printf( "Should take %d hours and %d minutes to finish\n", + $time_to_do, $mins ); +} + +while ( my $report = $reports->next ) { + $num_reports--; + next unless $report->latitude && $report->longitude; + next if $report->geocode; + + my $j = FixMyStreet::Geocode::Bing::reverse( $report->latitude, + $report->longitude, $bing_culture ); + + $report->geocode($j); + $report->update; + + print "$num_reports left to populate\n" unless $num_reports % 100; + sleep 10; +} + +print "done\n"; diff --git a/bin/fixmystreet.com/rotate-photos b/bin/fixmystreet.com/rotate-photos new file mode 100755 index 000000000..7b8109d65 --- /dev/null +++ b/bin/fixmystreet.com/rotate-photos @@ -0,0 +1,64 @@ +#!/usr/bin/perl -w + +# rotate-photos: +# Manaully fix any already-uploaded photos that are orientated wrongly. +# +# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: rotate-photos,v 1.1 2008-12-11 15:18:40 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use Digest::SHA qw(sha1_hex); + +use Utils; +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::TempFiles; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); +} + +my $r = select_all("select id, photo from problem where service='iPhone'"); +foreach (@$r) { + my $id = $_->{id}; + my $photo = $_->{photo}; + + if (length($photo) == 40) { + # If photo field contains a hash + my $filename = mySociety::Config::get('UPLOAD_DIR') . "$photo.jpeg"; + `jhead -autorot $filename`; + } else { + my ($fh, $filename) = mySociety::TempFiles::named_tempfile('.jpeg'); + print $fh $photo; + close $fh; + my $out = `jhead -autorot $filename`; + if ($out) { + open(FP, $filename) or die $!; + $photo = join('', ); + close FP; + my $fileid = sha1_hex($photo); + rename $filename, mySociety::Config::get('UPLOAD_DIR') . "$fileid.jpeg"; + dbh()->do('UPDATE problem SET photo=? WHERE id=?', {}, $fileid, $id); + dbh()->commit(); + } else { + unlink $filename; + } + } +} + diff --git a/bin/fixmystreet.com/showcouncilrates b/bin/fixmystreet.com/showcouncilrates new file mode 100755 index 000000000..1dacae597 --- /dev/null +++ b/bin/fixmystreet.com/showcouncilrates @@ -0,0 +1,74 @@ +#!/usr/bin/perl + +use warnings; +use strict; + +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use POSIX qw(strcoll); +use mySociety::DBHandle qw(dbh); + +use mySociety::MaPit; +use FixMyStreet::Geocode::OSM; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); +} + +my $query = "SELECT council, COUNT(*) AS total, SUM(fixed) AS fixed + FROM (SELECT council, + CASE WHEN state in ('fixed','fixed - user', 'fixed - council') + THEN 1 ELSE 0 END AS fixed + FROM problem WHERE confirmed IS NOT NULL AND + state IN ('fixed', 'fixed - user', 'fixed - council', 'confirmed') AND + whensent < NOW() - INTERVAL '4 weeks') AS a + GROUP BY council"; + +my $stats = dbh()->selectall_arrayref($query, { Slice => {} }); + +my @councils; +foreach my $row (@$stats) { + if ($row->{council}) { + $row->{council} =~ s/\|.*//g; + my @council_ids = split(/,/, $row->{council}); + push(@councils, @council_ids); + $row->{council} = \@council_ids; + } +} +my $areas_info = mySociety::MaPit::call('areas', \@councils); +my %adminsum; +my %adminfixed; +foreach my $row (@$stats){ + if ($row->{council}) { + for my $councilid (@{$row->{council}}) { + $adminsum{$councilid} += $row->{total}; + $adminfixed{$councilid} += $row->{fixed}; + } + } +} + +foreach my $councilid (sort sort_councils keys %adminsum) { + my $council = $areas_info->{$councilid}->{name}; + my $total = $adminsum{$councilid}; + my $fixed = $adminfixed{$councilid}; + printf("%5.1f %4d %s (%d)\n", 100 * $fixed / $total, $total, + $council, $councilid); +} + +sub sort_councils { + my $retval = ($adminfixed{$b} / $adminsum{$b}) <=> + ($adminfixed{$a} / $adminsum{$a}); + $retval = $adminsum{$b} <=> $adminsum{$a} unless $retval; + $retval = strcoll($areas_info->{$a}->{name}, + $areas_info->{$b}->{name}) unless $retval; + return $retval; +} diff --git a/bin/fixmystreet.com/update-areas b/bin/fixmystreet.com/update-areas new file mode 100755 index 000000000..a8cc01769 --- /dev/null +++ b/bin/fixmystreet.com/update-areas @@ -0,0 +1,46 @@ +#!/usr/bin/perl -w + +# update-areas: +# One-off script to populate the areas column of the problem table +# +# Copyright (c) 2007 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org +# +# $Id: update-areas,v 1.1 2007-08-24 12:27:30 matthew Exp $ + +use strict; +require 5.8.0; + +# Horrible boilerplate to set up appropriate library paths. +use FindBin; +use lib "$FindBin::Bin/../perllib"; +use lib "$FindBin::Bin/../commonlib/perllib"; + +use mySociety::Config; +use mySociety::DBHandle qw(dbh select_all); +use mySociety::MaPit; +use mySociety::VotingArea; + +BEGIN { + mySociety::Config::set_file("$FindBin::Bin/../conf/general"); + mySociety::DBHandle::configure( + Name => mySociety::Config::get('FMS_DB_NAME'), + User => mySociety::Config::get('FMS_DB_USER'), + Password => mySociety::Config::get('FMS_DB_PASS'), + Host => mySociety::Config::get('FMS_DB_HOST', undef), + Port => mySociety::Config::get('FMS_DB_PORT', undef) + ); +} + +print "Fetching problems...\n"; +my $ids = select_all("select id, latitude, longitude from problem where areas=''"); +print "Updating areas...\n"; +my $c = 0; +foreach (@$ids) { + my $areas = mySociety::MaPit::get_voting_areas_by_location({latitude=>$_->{latitude}, longitude=>$_->{longitude}}, 'polygon'); + $areas = ',' . join(',', sort keys %$areas) . ','; + dbh()->do('update problem set areas=? where id=?', {}, $areas, $_->{id}); + dbh()->commit(); + print "Done: " . (++$c) . "/" . @$ids . "\n"; +} + diff --git a/bin/fixmystreet.com/update-send-questionnaire b/bin/fixmystreet.com/update-send-questionnaire new file mode 100755 index 000000000..7a231b919 --- /dev/null +++ b/bin/fixmystreet.com/update-send-questionnaire @@ -0,0 +1,29 @@ +#!/usr/bin/env perl + +=head1 DESCRIPTION + +Ad hoc script to update the send_questionnaire column on the +reports in cobrands that don't send questionnaires at all. + +=cut + +use strict; +use warnings; + +use FixMyStreet::App; + +my @cobrands; +foreach my $cobrand ( FixMyStreet::Cobrand->available_cobrand_classes() ) { + next unless $cobrand->{class}; + my $cls = $cobrand->{class}->new(); + push @cobrands, $cls->moniker if !$cls->send_questionnaires(); +} + +my $problems = FixMyStreet::App->model('DB::Problem')->search({ + cobrand => \@cobrands, + send_questionnaire => 1, +}); + +$problems->update( { + send_questionnaire => 0 +}); diff --git a/bin/generate_council_location b/bin/generate_council_location deleted file mode 100755 index c7aea8074..000000000 --- a/bin/generate_council_location +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/perl - -use strict; -use JSON; -use LWP::Simple; - -my $id = shift; - -my $url = sprintf( 'http://mapit.mysociety.org/area/%s/geometry', $id); - -my $json = get( $url ); -my $geo = decode_json( $json ); - -my $max_lat = $geo->{max_lat}; -my $min_lat = $geo->{min_lat}; -my $max_lon = $geo->{max_lon}; -my $min_lon = $geo->{min_lon}; -my $c_lat = $geo->{centre_lat}; -my $c_lon = $geo->{centre_lon}; - -my $spn_lat = $max_lat - $min_lat; -my $spn_lon = $max_lon - $min_lon; - -print <<"EOT"; - centre => '$c_lat,$c_lon', - span => '$spn_lat,$spn_lon', - bounds => [ $min_lat, $min_lon, $max_lat, $max_lon ], -EOT diff --git a/bin/import-categories b/bin/import-categories deleted file mode 100755 index e9008b93f..000000000 --- a/bin/import-categories +++ /dev/null @@ -1,78 +0,0 @@ -#!/usr/bin/perl -w - -# import-categories: -# Initial choice of categories for councils. Most likely wrong. -# -# Copyright (c) 2007 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: import-categories,v 1.3 2007-08-02 11:44:59 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::MaPit; -use mySociety::VotingArea; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); -} - -my @district_cats = ( - 'Abandoned vehicles', 'Car parking', 'Dog fouling', 'Flyposting', 'Flytipping', - 'Graffiti', 'Parks/landscapes', 'Public toilets', 'Rubbish (refuse and recycling)', - 'Street nameplates', 'Street cleaning', 'Trees' -); -my @county_cats = ( - 'Bus stops', 'Pavements/footpaths', 'Potholes', 'Roads/highways', - 'Road traffic signs', 'Street lighting', 'Traffic lights' -); - -my @councils; -foreach my $type (@$mySociety::VotingArea::council_parent_types) { - my $areas = mySociety::MaPit::get_areas_by_type($type); - push @councils, @$areas; -} -my $councils = mySociety::MaPit::get_voting_areas_info(\@councils); -foreach my $id (keys %$councils) { - my $type = $councils->{$id}->{type}; - my ($email,$confirmed) = dbh()->selectrow_array( - "SELECT email,confirmed FROM contacts WHERE deleted='f' - and area_id=? AND category='Other'", {}, $id); - next unless $email; - if ($type eq 'DIS') { - add_categories($id, $email, $confirmed, @district_cats); - } elsif ($type eq 'CTY') { - add_categories($id, $email, $confirmed, @county_cats); - } else { - add_categories($id, $email, $confirmed, @district_cats, @county_cats); - } -} -dbh()->commit(); - -sub add_categories { - my ($id, $email, $confirmed, @cats) = @_; - foreach (@cats) { - dbh()->do("insert into contacts - (area_id, category, email, editor, whenedited, note, confirmed, deleted) - values - (?, ?, ?, 'import', ms_current_timestamp(), 'Initial copy', ?, 'f')", {}, - $id, $_, $email, ($confirmed ? 1 : 0) - ); - } -} - diff --git a/bin/install-as-user b/bin/install-as-user index ef97a41b3..33a74c140 100755 --- a/bin/install-as-user +++ b/bin/install-as-user @@ -134,7 +134,7 @@ echo $DONE_MSG # Generate po and mo files (these invocations taken from Kagee's script): echo "Creating locale .mo files" bin/cron-wrapper bin/make_po FixMyStreet-EmptyHomes -bin/cron-wrapper bin/make_emptyhomes_welsh_po +bin/cron-wrapper bin/emptyhomes/make_welsh_po commonlib/bin/gettext-makemo FixMyStreet echo $DONE_MSG diff --git a/bin/kasabi b/bin/kasabi deleted file mode 100755 index 456b2f4d1..000000000 --- a/bin/kasabi +++ /dev/null @@ -1,234 +0,0 @@ -#!/usr/bin/env python - -import sys -import datetime -import json -import os.path -import re -import urllib -import yaml - -import pytassium -import psycopg2 -import psycopg2.extras -from rdfchangesets import BatchChangeSet -from rdflib.namespace import XSD - -# Set up data access -config = yaml.load(open(os.path.abspath(os.path.join(os.path.dirname(__file__), '../conf/general.yml')))) -dataset = pytassium.Dataset('fixmystreet', config['KASABI_API_KEY']) -db = psycopg2.connect( "host='{host}' dbname='{name}' user='{user}' password='{password}'".format( - host=config['FMS_DB_HOST'], - name=config['FMS_DB_NAME'], - user=config['FMS_DB_USER'], - password=config['FMS_DB_PASS'] -)) -cursor = db.cursor(cursor_factory=psycopg2.extras.RealDictCursor) -report_cursor = db.cursor(cursor_factory=psycopg2.extras.RealDictCursor) - -def main(): - - # Check the status of our dataset - response, status = dataset.status() - if response.status not in range(200, 300) or status['storageMode'] == 'read-only': - # We can't import anything, so let's not bother - sys.exit() - - # Fetch reports that have changed since last update in dataset - response, data = dataset.select('select (max(?lastupdate) as ?max) where { ?report ?lastupdate }') - max_lastUpdate = data[1][0]['max'] - query = """ - SELECT id, latitude, longitude, used_map, council, - category, title, detail, (photo IS NOT NULL) as photo, - confirmed, lastupdate, whensent, state - FROM problem - WHERE state not in ('unconfirmed', 'partial') - """ - if len(sys.argv) > 1 and sys.argv[1].isdigit(): - cursor.execute("%s AND id=%%s" % query, (sys.argv[1],)) - else: - cursor.execute("%s AND lastupdate > %%s ORDER BY lastupdate" % query, (str(max_lastUpdate),)) - - for report in cursor: - changeset = FixMyStreetChangeSet(dataset) - if report['state'] == 'hidden': - # If the report has been hidden, just remove it - changeset.remove_report(report) - else: - - # Canonicalise some values - report['latitude'] = round(report['latitude'], 6) # <10cm - report['longitude'] = round(report['longitude'], 6) - report['title'] = tidy_string(report['title']) - report['detail'] = tidy_string(report['detail']) - report['confirmed'] = report['confirmed'].replace(microsecond=0).isoformat() # Don't want microseconds - report['lastupdate'] = report['lastupdate'].replace(microsecond=0).isoformat() - report['council'] = sorted(re.sub('\|.*', '', report['council'] or '').split(',')) # Remove missing councils - - # Fetch updates to note state changes - states = [ { 'state': 'confirmed', 'time': report['confirmed'] } ] - report_cursor.execute(""" - SELECT id, mark_fixed, mark_open, problem_state, confirmed - FROM comment - WHERE problem_id=%s AND state='confirmed' - ORDER BY created - """, (report['id'], )) - for update in report_cursor: - t = update['confirmed'].replace(microsecond=0).isoformat() - if update['problem_state']: - states.append( { 'state': update['problem_state'], 'time': t } ) - elif update['mark_fixed']: - states.append( { 'state': 'fixed - user', 'time': t } ) - elif update['mark_open']: - states.append( { 'state': 'confirmed', 'time': t } ) - - # Remove and then re-add the report - changeset.remove_report(report) - changeset.add_report(report, states) - changeset.apply() - -# Escape double quotes and backslashes, remove carriage returns -def tidy_string(s): - return s.replace('\r', '').replace('\\', '\\\\').replace('"', r'\"') - -class FixMyStreetChangeSet(object): - """Something that hosts either or both of a BatchChangeSet and a Turtle - string for sending to Kasabi. Changes are done by removing all triples - and then readding the report.""" - _changeset = None - data = '' - - def __init__(self, dataset): - self.dataset = dataset - - def __str__(self): - return unicode(self).encode('utf-8') - - def __unicode__(self): - g = self.changeset.getGraph() - data = g.serialize(format='xml') - return "Changeset:\n" + data + "\nNew data:\n" + self.data - - @property - def changeset(self): - if not self._changeset: - self._changeset = BatchChangeSet() - self._changeset.setChangeReason("Report updates") - self._changeset.setCreatorName("FixMyStreet") - return self._changeset - - def apply(self): - if len(self.changeset.changesets): - #response, data = self.dataset.apply_changeset(self.changeset) - # XXX Do everything the above call does, but additionally escape carriage returns to prevent 409 error - api = self.dataset.get_api('update') - g = self.changeset.getGraph() - data = g.serialize(format='xml') - data = data.replace('\r', ' ') - response, data = api.client.request(api.uri, "POST", body=data, headers={"accept" : "*/*", 'content-type':'application/vnd.talis.changeset+xml', 'X_KASABI_APIKEY':api.apikey}) - if response.status not in range(200, 300): - print 'Error:', response.status, response.reason, data - if self.data: - response, data = self.dataset.store_data(self.data, media_type='text/turtle') - if response.status not in range(200, 300): - print 'Error:', response.status, response.reason, data - - def remove_report(self, report): - uri = 'http://data.kasabi.com/dataset/fixmystreet/report/{id}'.format(**report) - response, data = self.dataset.select('select ?p ?o where {{ <{0}> ?p ?o }}'.format(uri)) - for row in data[1]: - # Need to set the datatype correctly for the lastUpdate - if str(row['p']) == 'http://data.kasabi.com/dataset/fixmystreet/def/lastUpdate': - row['o'].datatype = XSD.dateTime - # Delete the referenced statuses - if re.match('http://data.kasabi.com/dataset/fixmystreet/report/\d+/status/\d+$', unicode(row['o'])): - uri2 = unicode(row['o']) - response2, data2 = self.dataset.select('select ?p ?o where {{ <{0}> ?p ?o }}'.format(uri2)) - for row2 in data2[1]: - self.changeset.remove(uri2, row2['p'], row2['o']) - self.changeset.remove(uri, row['p'], row['o']) - - def add_report(self, report, states): - # Work out the update states - c = 0 - state_data = { 'refs': '', 'objs': '' } - for state in states: - state_data['refs'] += ' ; fixmystreet:status \n'.format(id=report['id'], c=c) - obj = re.sub('[ -]', '', ' '.join(x.capitalize() for x in state['state'].split())) - if obj == 'Confirmed': obj = 'Open' - state_data['objs'] += """ a fixmystreet:{state}Status - ; event:time - . -""".format( id=report['id'], c=c, state=obj, time=state['time'] ) - # ; rdfs:label - c += 1 - - # Get info for the councils - council_data = { 'sentTo': '', 'areaNames': [] } - for council in report['council']: - if not council: continue - js = json.load(urllib.urlopen('http://mapit.mysociety.org/area/{0}'.format(council))) - os_id = int(js['codes']['unit_id']) + 7000000000000000 - if report['whensent']: - council_data['sentTo'] += ' ; fixmystreet:sentTo \n'.format(os_id=os_id) - council_data['areaNames'].append(js['name']) - council_data.setdefault('firstCouncil', council) - council_data['areaNames'] = ' / '.join(council_data['areaNames']) - council_data.setdefault('firstCouncil', '0') - -# easting/northing - - self.data += ''' -@prefix fixmystreet: . -@prefix dct: . -@prefix event: . -@prefix geo: . -@prefix xsd: . -@prefix skos: . -@prefix foaf: . -@prefix georss: . -@prefix owl: . - - a fixmystreet:Report - ; fixmystreet:location - ; dct:description """{detail}""" - ; dct:title "{title}" -{photo_url} -{state_data[refs]} -{council_data[sentTo]} - ; fixmystreet:category - ; fixmystreet:lastUpdate "{lastupdate}"^^xsd:dateTime - ; foaf:page - . - - a fixmystreet:Location - ; geo:lat "{latitude}" - ; geo:long "{longitude}" - ; georss:point "{latitude} {longitude}" - . - - - owl:sameAs - . -{state_data[objs]} - a skos:Concept - ; skos:prefLabel "{category}" - ; skos:altLabel "{category} in {council_data[areaNames]}" - . - '''.format( - photo_url = ' ; foaf:depiction '.format(**report) if report['photo'] else '', - state_data = state_data, - council_data = council_data, - category_uri = report['category'].lower().replace(' ', '-'), - **report - ) - -# ; skos:broader - -# this category is the broadest highlevel street light category -# a skos:Concept -# ; skos:prefLabel "Street lights" -# . - -main() - diff --git a/bin/load-contacts b/bin/load-contacts deleted file mode 100755 index b18699db1..000000000 --- a/bin/load-contacts +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/perl -w - -# canonicalise-csv: -# Convert provided CSV file into one with standard names for MaPit -# -# Copyright (c) 2006 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: load-contacts,v 1.3 2007-08-02 11:44:59 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::Random; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); - - if (!dbh()->selectrow_array('select secret from secret for update of secret')) { - local dbh()->{HandleError}; - dbh()->do('insert into secret (secret) values (?)', {}, unpack('h*', mySociety::Random::random_bytes(32))); - } - dbh()->commit(); -} - -open(FP, "$FindBin::Bin/../data/councils_canonical.csv"); -while () { - s/\r?\n//g; - my ($id, $email) = split /,/; - dbh()->do("INSERT INTO contacts (area_id, email, editor, whenedited, note) - VALUES (?, ?, 'import', ms_current_timestamp(), 'Initial import')", - {}, $id, $email); -} -dbh()->commit(); -close(FP); diff --git a/bin/load-norwegian-contacts b/bin/load-norwegian-contacts deleted file mode 100755 index b73778848..000000000 --- a/bin/load-norwegian-contacts +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/perl -w - -# load-norwegian-contacts: -# Load initial contact list into fiksgatami. -# The format is -# Name;ID;email-address;Category1,Category2,... -# -# Based on script load-contacts copyright (c) 2006 UK Citizens Online Democracy. -# Copyright 2011 Petter Reinholdtsen -# -# $Id: load-norwegian-contacts,v 1.0 2007-08-02 11:44:59 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::Random; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); - - if (!dbh()->selectrow_array('select secret from secret for update of secret')) { - local dbh()->{HandleError}; - dbh()->do('insert into secret (secret) values (?)', {}, unpack('h*', mySociety::Random::random_bytes(32))); - } - dbh()->commit(); -} - -my $datafile = shift; -my $defcategories; -open(FP, "<", $datafile) or die "Unable to read from $datafile"; -while () { - s/\r?\n//g; - print "Loading $_\n"; - my ($id, $name, $email, $categories) = split /;/; - if (0 == $id) { - $defcategories = $categories; - next; - } - unless ($categories) { - $categories = $defcategories; - } -# dbh()->do("INSERT INTO contacts (area_id, email, editor, whenedited, note, confirmed, deleted) -# VALUES (?, ?, 'import', ms_current_timestamp(), 'Initial import', 'false', 'false')", -# {}, $id, $email); -# } else { - for my $category (split(/,\s*/, $categories)) { - print " Category '$category'\n"; - dbh()->do("INSERT INTO contacts (area_id, email, category, editor, whenedited, note, confirmed, deleted) - VALUES (?, ?, ?, 'import', ms_current_timestamp(), 'Initial import', 'true', 'false')", - {}, $id, $email, $category); - } -# } -} -dbh()->commit(); -close(FP); diff --git a/bin/make_emptyhomes_welsh_po b/bin/make_emptyhomes_welsh_po deleted file mode 100755 index f4f6850d6..000000000 --- a/bin/make_emptyhomes_welsh_po +++ /dev/null @@ -1,114 +0,0 @@ -#!/usr/bin/perl -w -use strict; - -# Given a translation of the Empty Homes version of the FixMyStreet .po file, -# fix it all up to go direct from FixMyStreet -> EH translation. - -use POSIX; -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use PoChange; - -chdir("$FindBin::Bin/../locale"); - -# First read in translation and match up. -open(INPO, 'cy_GB.UTF-8/LC_MESSAGES/EmptyHomes.po') or die $!; - -my $state = 'start'; -my $msgid = ''; -my $fuzzy = 0; -my $msgstr; -my %lookup; -while () { - if (m/^#, fuzzy/) { - $fuzzy = 1; - } elsif (m/^#/) { - # comment or blank line - } elsif (m/^\s+$/) { - # blank line separates translations - $msgid =~ s/"\n"//g; - $lookup{$msgid} = [ $msgstr, $fuzzy ]; - $state = 'msgid'; - $msgid = ""; - $fuzzy = 0; - } elsif ($state eq 'msgid' && (m/^msgstr "/ || m/^msgstr\[0\] "/)) { - $msgstr = $_; - $state = 'msgstr'; - } elsif ($state eq 'msgstr') { - $msgstr .= $_; - } elsif ($state eq 'msgid') { - $msgid .= $_; - } -} -$lookup{$msgid} = [ $msgstr, $fuzzy ]; -close INPO; - -mkdir("en_GB.UTF-8"); -mkdir("en_GB.UTF-8/LC_MESSAGES"); -mkdir("cy_GB.UTF-8"); -mkdir("cy_GB.UTF-8/LC_MESSAGES"); - -open(MAINPO, 'FixMyStreet.po') or die; -open(OUTPO, ">cy_GB.UTF-8/LC_MESSAGES/FixMyStreet-EmptyHomes.po") or die; - -print OUTPO "# AUTOMATICALLY GENERATED by make_emptyhomes_welsh_po, do not edit\n\n"; - -my $buffer = ""; -my $start = 0; -while() { - if (!$start) { - s/#, fuzzy/#/; - } - if (m/"Last-Translator: FULL NAME/) { - $_ = '"Last-Translator: mysociety/bin/make_emptyhomes_po\\n"'."\n"; - } - if (m/"PO-Revision-Date: YEAR-MO-DA/) { - my $time = POSIX::strftime("%Y-%m-%d %H:%M%z", localtime(time())); - $_ = '"PO-Revision-Date: '.$time.'\\n"'."\n"; - } - if (m/"Language-Team: LANGUAGE/) { - $_ = '"Language-Team: mySociety\\n"'."\n"; - } - if (m/"Plural-Forms: nplurals=/) { - $_ = '"Plural-Forms: nplurals=2; plural=n != 1;\\n"'."\n"; - } - - if (m/^#/) { - # comment or blank line - print OUTPO $_; - } elsif (m/^\s+$/) { - # blank line - $start = 1; - $buffer = ""; - print OUTPO $_; - } elsif ($start && (m/^msgstr "/ || m/^msgstr\[0\] "/)) { - # start of translated text - translate English into Empty Homes language - my $new_buffer = PoChange::fixmystreet_to_reportemptyhomes($buffer); - $new_buffer =~ s/"\n"//g; - - if ($lookup{$new_buffer} && $lookup{$new_buffer}[1]) { - print OUTPO "#, fuzzy\n"; - } - - print OUTPO $buffer; - if ($lookup{$new_buffer}) { - print OUTPO $lookup{$new_buffer}[0]; - } else { - if (m/^msgstr\[0\] ""/) { - $new_buffer =~ s/^msgid "/msgstr[0] "/m; - $new_buffer =~ s/^msgid_plural "/msgstr[1] "/m; - $_ = ; # skip untranslated plural - } else { - $new_buffer =~ s/^msgid "/msgstr "/; - } - print OUTPO $new_buffer; - } - - $buffer = ""; - } elsif (!$start) { - print OUTPO $_; - } else { - $buffer .= $_; - } -} - diff --git a/bin/populate_bing_cache b/bin/populate_bing_cache deleted file mode 100755 index 17c8911d0..000000000 --- a/bin/populate_bing_cache +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/perl - -use strict; -use warnings; -require 5.8.0; - -use Data::Dumper; - -use FixMyStreet::App; -use FixMyStreet::Geocode::Bing; - -my $bing_culture = 'en-GB'; - -my $reports = FixMyStreet::App->model('DB::Problem')->search( - { - geocode => undef, - confirmed => { '!=', undef }, - latitude => { '!=', 0 }, - longitude => { '!=', 0 }, - }, - { - select => [qw/id geocode confirmed latitude longitude/], - order_by => { -desc => 'confirmed' } - } -); - -my $num_reports = $reports->count(); -print "Found $num_reports lacking geocode information\n"; - -my $time_to_do = ( $num_reports * 10 ) / 60 / 60; -if ( $time_to_do > 24 ) { - my $days = $time_to_do / 24; - my $hours = $time_to_do % 24; - printf( "Should take %d days and %d hours to finish\n", $days, $hours ); -} -elsif ( $time_to_do < 1 ) { - printf( "Should take %d minutes to finish\n", $time_to_do * 60 ); -} -else { - my $mins = ( $num_reports * 10 ) / 60 % 60; - printf( "Should take %d hours and %d minutes to finish\n", - $time_to_do, $mins ); -} - -while ( my $report = $reports->next ) { - $num_reports--; - next unless $report->latitude && $report->longitude; - next if $report->geocode; - - my $j = FixMyStreet::Geocode::Bing::reverse( $report->latitude, - $report->longitude, $bing_culture ); - - $report->geocode($j); - $report->update; - - print "$num_reports left to populate\n" unless $num_reports % 100; - sleep 10; -} - -print "done\n"; diff --git a/bin/rotate-photos b/bin/rotate-photos deleted file mode 100755 index 7b8109d65..000000000 --- a/bin/rotate-photos +++ /dev/null @@ -1,64 +0,0 @@ -#!/usr/bin/perl -w - -# rotate-photos: -# Manaully fix any already-uploaded photos that are orientated wrongly. -# -# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: rotate-photos,v 1.1 2008-12-11 15:18:40 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use Digest::SHA qw(sha1_hex); - -use Utils; -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::TempFiles; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); -} - -my $r = select_all("select id, photo from problem where service='iPhone'"); -foreach (@$r) { - my $id = $_->{id}; - my $photo = $_->{photo}; - - if (length($photo) == 40) { - # If photo field contains a hash - my $filename = mySociety::Config::get('UPLOAD_DIR') . "$photo.jpeg"; - `jhead -autorot $filename`; - } else { - my ($fh, $filename) = mySociety::TempFiles::named_tempfile('.jpeg'); - print $fh $photo; - close $fh; - my $out = `jhead -autorot $filename`; - if ($out) { - open(FP, $filename) or die $!; - $photo = join('', ); - close FP; - my $fileid = sha1_hex($photo); - rename $filename, mySociety::Config::get('UPLOAD_DIR') . "$fileid.jpeg"; - dbh()->do('UPDATE problem SET photo=? WHERE id=?', {}, $fileid, $id); - dbh()->commit(); - } else { - unlink $filename; - } - } -} - diff --git a/bin/showcouncilrates b/bin/showcouncilrates deleted file mode 100755 index 1dacae597..000000000 --- a/bin/showcouncilrates +++ /dev/null @@ -1,74 +0,0 @@ -#!/usr/bin/perl - -use warnings; -use strict; - -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use POSIX qw(strcoll); -use mySociety::DBHandle qw(dbh); - -use mySociety::MaPit; -use FixMyStreet::Geocode::OSM; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); -} - -my $query = "SELECT council, COUNT(*) AS total, SUM(fixed) AS fixed - FROM (SELECT council, - CASE WHEN state in ('fixed','fixed - user', 'fixed - council') - THEN 1 ELSE 0 END AS fixed - FROM problem WHERE confirmed IS NOT NULL AND - state IN ('fixed', 'fixed - user', 'fixed - council', 'confirmed') AND - whensent < NOW() - INTERVAL '4 weeks') AS a - GROUP BY council"; - -my $stats = dbh()->selectall_arrayref($query, { Slice => {} }); - -my @councils; -foreach my $row (@$stats) { - if ($row->{council}) { - $row->{council} =~ s/\|.*//g; - my @council_ids = split(/,/, $row->{council}); - push(@councils, @council_ids); - $row->{council} = \@council_ids; - } -} -my $areas_info = mySociety::MaPit::call('areas', \@councils); -my %adminsum; -my %adminfixed; -foreach my $row (@$stats){ - if ($row->{council}) { - for my $councilid (@{$row->{council}}) { - $adminsum{$councilid} += $row->{total}; - $adminfixed{$councilid} += $row->{fixed}; - } - } -} - -foreach my $councilid (sort sort_councils keys %adminsum) { - my $council = $areas_info->{$councilid}->{name}; - my $total = $adminsum{$councilid}; - my $fixed = $adminfixed{$councilid}; - printf("%5.1f %4d %s (%d)\n", 100 * $fixed / $total, $total, - $council, $councilid); -} - -sub sort_councils { - my $retval = ($adminfixed{$b} / $adminsum{$b}) <=> - ($adminfixed{$a} / $adminsum{$a}); - $retval = $adminsum{$b} <=> $adminsum{$a} unless $retval; - $retval = strcoll($areas_info->{$a}->{name}, - $areas_info->{$b}->{name}) unless $retval; - return $retval; -} diff --git a/bin/update-areas b/bin/update-areas deleted file mode 100755 index a8cc01769..000000000 --- a/bin/update-areas +++ /dev/null @@ -1,46 +0,0 @@ -#!/usr/bin/perl -w - -# update-areas: -# One-off script to populate the areas column of the problem table -# -# Copyright (c) 2007 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: update-areas,v 1.1 2007-08-24 12:27:30 matthew Exp $ - -use strict; -require 5.8.0; - -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - -use mySociety::Config; -use mySociety::DBHandle qw(dbh select_all); -use mySociety::MaPit; -use mySociety::VotingArea; - -BEGIN { - mySociety::Config::set_file("$FindBin::Bin/../conf/general"); - mySociety::DBHandle::configure( - Name => mySociety::Config::get('FMS_DB_NAME'), - User => mySociety::Config::get('FMS_DB_USER'), - Password => mySociety::Config::get('FMS_DB_PASS'), - Host => mySociety::Config::get('FMS_DB_HOST', undef), - Port => mySociety::Config::get('FMS_DB_PORT', undef) - ); -} - -print "Fetching problems...\n"; -my $ids = select_all("select id, latitude, longitude from problem where areas=''"); -print "Updating areas...\n"; -my $c = 0; -foreach (@$ids) { - my $areas = mySociety::MaPit::get_voting_areas_by_location({latitude=>$_->{latitude}, longitude=>$_->{longitude}}, 'polygon'); - $areas = ',' . join(',', sort keys %$areas) . ','; - dbh()->do('update problem set areas=? where id=?', {}, $areas, $_->{id}); - dbh()->commit(); - print "Done: " . (++$c) . "/" . @$ids . "\n"; -} - diff --git a/bin/update-send-questionnaire b/bin/update-send-questionnaire deleted file mode 100755 index 7a231b919..000000000 --- a/bin/update-send-questionnaire +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env perl - -=head1 DESCRIPTION - -Ad hoc script to update the send_questionnaire column on the -reports in cobrands that don't send questionnaires at all. - -=cut - -use strict; -use warnings; - -use FixMyStreet::App; - -my @cobrands; -foreach my $cobrand ( FixMyStreet::Cobrand->available_cobrand_classes() ) { - next unless $cobrand->{class}; - my $cls = $cobrand->{class}->new(); - push @cobrands, $cls->moniker if !$cls->send_questionnaires(); -} - -my $problems = FixMyStreet::App->model('DB::Problem')->search({ - cobrand => \@cobrands, - send_questionnaire => 1, -}); - -$problems->update( { - send_questionnaire => 0 -}); diff --git a/bin/zerotb/import_clinic_list.pl b/bin/zerotb/import_clinic_list.pl new file mode 100755 index 000000000..359a63925 --- /dev/null +++ b/bin/zerotb/import_clinic_list.pl @@ -0,0 +1,115 @@ +#!/usr/bin/perl + +use strict; +use FixMyStreet::App; +use Text::CSV; +use Getopt::Long::Descriptive; + +use constant TITLE => 0; +use constant DESC => 1; +use constant LATLONG => 2; +use constant EMAIL => 3; + +my ($opt, $usage) = describe_options( + '%c %o', + ['file|f=s', "path to csv file with list of clinics", { required => 1 } ], + ['email|e=s', "default email address for the updates to be sent to", { required => 1 } ], + ['verbose|v', "print out all services as they are found"], + ['help', "print usage message and exit" ], +); +print($usage->text), exit if $opt->help; + +my $csv = Text::CSV->new ( { binary => 1 } ) # should set binary attribute. + or die "Cannot use CSV: ".Text::CSV->error_diag (); +open my $fh, "<:encoding(utf8)", $opt->file or die "Failed to open " . $opt->file . ": $!"; + +my $clinic_user = FixMyStreet::App->model('DB::User')->find_or_create({ + email => $opt->email +}); +if ( not $clinic_user->in_storage ) { + $clinic_user->insert; +} + +# throw away header line +my $title_row = $csv->getline( $fh ); + +while ( my $row = $csv->getline( $fh ) ) { + my $clinics = FixMyStreet::App->model('DB::Problem')->search({ + title => $row->[TITLE] + }); + + my ($lat, $long) = split(',', $row->[LATLONG]); + my $p; + my $count = $clinics->count; + if ( $count == 0 ) { + $p = FixMyStreet::App->model('DB::Problem')->create({ + title => $row->[TITLE], + latitude => $lat, + longitude => $long, + used_map => 1, + anonymous => 1, + state => 'unconfirmed', + name => '', + user => $clinic_user, + detail => '', + areas => '', + postcode => '' + }); + } elsif ( $count == 1 ) { + $p = $clinics->first; + } else { + printf "Too many matches for: %s\n", $row->[TITLE]; + next; + } + $p->detail( $row->[DESC] ); + $p->latitude( $lat ); + $p->longitude( $long ); + $p->confirm; + + if ( $p->in_storage ) { + printf( "Updating entry for %s\n", $row->[TITLE] ) if $opt->verbose; + $p->update; + } else { + printf( "Creating entry for %s\n", $row->[TITLE] ) if $opt->verbose; + $p->insert; + } + $p->discard_changes; + + # disabling existing alerts in case email addresses have changed + my $existing = FixMyStreet::App->model('DB::Alert')->search({ + alert_type => 'new_updates', + parameter => $p->id + }); + $existing->update( { confirmed => 0 } ); + + if ( $row->[EMAIL] ) { + my $u = FixMyStreet::App->model('DB::User')->find_or_new({ + email => $row->[EMAIL] + }); + $u->insert unless $u->in_storage; + create_update_alert( $u, $p, $opt->verbose ); + } + + create_update_alert( $clinic_user, $p, $opt->verbose ); +} + +sub create_update_alert { + my ( $user, $p, $verbose ) = @_; + my $a = FixMyStreet::App->model('DB::Alert')->find_or_new({ + alert_type => 'new_updates', + user => $user, + parameter => $p->id, + }); + + $a->confirmed(1); + + if ( $a->in_storage ) { + printf( "Updating update alert for %s on %s\n", $user->email, $p->title ) + if $verbose; + $a->update; + } else { + printf( "Creating update alert for %s on %s\n", $user->email, $p->title ) + if $verbose; + $a->insert; + } +} diff --git a/bin/zerotb_import_clinic_list.pl b/bin/zerotb_import_clinic_list.pl deleted file mode 100755 index 359a63925..000000000 --- a/bin/zerotb_import_clinic_list.pl +++ /dev/null @@ -1,115 +0,0 @@ -#!/usr/bin/perl - -use strict; -use FixMyStreet::App; -use Text::CSV; -use Getopt::Long::Descriptive; - -use constant TITLE => 0; -use constant DESC => 1; -use constant LATLONG => 2; -use constant EMAIL => 3; - -my ($opt, $usage) = describe_options( - '%c %o', - ['file|f=s', "path to csv file with list of clinics", { required => 1 } ], - ['email|e=s', "default email address for the updates to be sent to", { required => 1 } ], - ['verbose|v', "print out all services as they are found"], - ['help', "print usage message and exit" ], -); -print($usage->text), exit if $opt->help; - -my $csv = Text::CSV->new ( { binary => 1 } ) # should set binary attribute. - or die "Cannot use CSV: ".Text::CSV->error_diag (); -open my $fh, "<:encoding(utf8)", $opt->file or die "Failed to open " . $opt->file . ": $!"; - -my $clinic_user = FixMyStreet::App->model('DB::User')->find_or_create({ - email => $opt->email -}); -if ( not $clinic_user->in_storage ) { - $clinic_user->insert; -} - -# throw away header line -my $title_row = $csv->getline( $fh ); - -while ( my $row = $csv->getline( $fh ) ) { - my $clinics = FixMyStreet::App->model('DB::Problem')->search({ - title => $row->[TITLE] - }); - - my ($lat, $long) = split(',', $row->[LATLONG]); - my $p; - my $count = $clinics->count; - if ( $count == 0 ) { - $p = FixMyStreet::App->model('DB::Problem')->create({ - title => $row->[TITLE], - latitude => $lat, - longitude => $long, - used_map => 1, - anonymous => 1, - state => 'unconfirmed', - name => '', - user => $clinic_user, - detail => '', - areas => '', - postcode => '' - }); - } elsif ( $count == 1 ) { - $p = $clinics->first; - } else { - printf "Too many matches for: %s\n", $row->[TITLE]; - next; - } - $p->detail( $row->[DESC] ); - $p->latitude( $lat ); - $p->longitude( $long ); - $p->confirm; - - if ( $p->in_storage ) { - printf( "Updating entry for %s\n", $row->[TITLE] ) if $opt->verbose; - $p->update; - } else { - printf( "Creating entry for %s\n", $row->[TITLE] ) if $opt->verbose; - $p->insert; - } - $p->discard_changes; - - # disabling existing alerts in case email addresses have changed - my $existing = FixMyStreet::App->model('DB::Alert')->search({ - alert_type => 'new_updates', - parameter => $p->id - }); - $existing->update( { confirmed => 0 } ); - - if ( $row->[EMAIL] ) { - my $u = FixMyStreet::App->model('DB::User')->find_or_new({ - email => $row->[EMAIL] - }); - $u->insert unless $u->in_storage; - create_update_alert( $u, $p, $opt->verbose ); - } - - create_update_alert( $clinic_user, $p, $opt->verbose ); -} - -sub create_update_alert { - my ( $user, $p, $verbose ) = @_; - my $a = FixMyStreet::App->model('DB::Alert')->find_or_new({ - alert_type => 'new_updates', - user => $user, - parameter => $p->id, - }); - - $a->confirmed(1); - - if ( $a->in_storage ) { - printf( "Updating update alert for %s on %s\n", $user->email, $p->title ) - if $verbose; - $a->update; - } else { - printf( "Creating update alert for %s on %s\n", $user->email, $p->title ) - if $verbose; - $a->insert; - } -} diff --git a/bin/zurich-overdue-alert b/bin/zurich-overdue-alert deleted file mode 100755 index c09aef1e2..000000000 --- a/bin/zurich-overdue-alert +++ /dev/null @@ -1,83 +0,0 @@ -#!/usr/bin/env perl - -# zurich-overdue-alert: -# Send email alerts to administrators for overdue admin activities. -# -# Copyright (c) 2012 UK Citizens Online Democracy. All rights reserved. -# Email: matthew@mysociety.org. WWW: http://www.mysociety.org - -use strict; -use warnings; -require 5.8.0; - -use DateTime; -use CronFns; -use FixMyStreet::App; - -my ($verbose, $nomail) = CronFns::options(); - -# Only run on working days -my $now = DateTime->now(); -exit if FixMyStreet::Cobrand::Zurich::is_public_holiday($now) or FixMyStreet::Cobrand::Zurich::is_weekend($now); - -my $cobrand = FixMyStreet::Cobrand->get_class_for_moniker('zurich')->new(); -my %bodies = map { $_->id => $_ } FixMyStreet::App->model("DB::Body")->all; - -loop_through( 'alert-moderation-overdue.txt', 0, 1, [ 'unconfirmed' ] ); -loop_through( 'alert-overdue.txt', 1, 6, 'in progress' ); -loop_through( 'alert-overdue.txt', 0, 6, ['confirmed', 'planned'] ); - -sub loop_through { - my ( $template, $include_parent, $days, $states ) = @_; - my $dtf = FixMyStreet::App->model("DB")->storage->datetime_parser; - my $date_threshold = $dtf->format_datetime(FixMyStreet::Cobrand::Zurich::sub_days( $now, $days )); - - my $reports = FixMyStreet::App->model("DB::Problem")->search( { - state => $states, - created => { '<', $date_threshold }, - bodies_str => { '!=', undef }, - } ); - - my %to_send = (); - while (my $row = $reports->next) { - $to_send{$row->bodies_str} .= '* ' . $row->id . ": '" . $row->title . "'\n\n"; - } - - my $template_path = FixMyStreet->path_to( "templates", "email", "zurich", $template )->stringify; - $template = Utils::read_file( $template_path ); - - foreach my $body_id (keys %to_send) { - send_alert( $template, $body_id, $to_send{$body_id}, $include_parent ); - } -} - -sub send_alert { - my ( $template, $body_id, $data, $include_parent ) = @_; - - my $body = $bodies{$body_id}; - my $body_email = $body->endpoint; - - my $h = { - data => $data, - admin_url => $cobrand->admin_base_url, - }; - - my $to = [ [ $body_email, $body->name ] ]; - if ( $include_parent ) { - my $parent = $body->parent; - my $parent_email = $parent->endpoint; - push @$to, [ $parent_email, $parent->name ]; - } - - FixMyStreet::App->send_email_cron( - { - _template_ => $template, - _parameters_ => $h, - To => $to, - From => [ FixMyStreet->config('CONTACT_EMAIL'), FixMyStreet->config('CONTACT_NAME') ], - }, - FixMyStreet->config('CONTACT_EMAIL'), - $nomail - ); -} - diff --git a/bin/zurich/geocode b/bin/zurich/geocode deleted file mode 100755 index 9482b27e6..000000000 --- a/bin/zurich/geocode +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/perl - -=head1 NAME - -zurich/geocode - commandline tool to test the Zurich geocoder - -=head1 SYNOPSIS - - # Firstly: - ## copy the GEOCODER config from a current Zurich conf to your conf/general.yml - $ eval `perl setenv.pl` - - $ bin/zurich/geocode Magnus - - # ... output from geocoder - -This can be used to test the results of, e.g. - - https://www.zueriwieneu.ch/ajax/geocode?term=Magnus - -but without the caching which FixMyStreet applies, and passing on any 500 -errors from the server. - -=cut - -use strict; -use warnings; -require 5.8.0; - - -use Data::Dumper; -use feature 'say'; - -use FixMyStreet; -use FixMyStreet::App; -use FixMyStreet::Geocode::Zurich; - -# TODO use FixMyStreet::override_config to get data from conf/general.yml.zurich if available -my $geocoder = FixMyStreet->config('GEOCODER') - or die "No GEOCODER config -- please copy appropriate Zurich conf to conf/general.yml"; - -my $c = FixMyStreet::App->new(); -my $s = join ' ', @ARGV; - -say Dumper( FixMyStreet::Geocode::Zurich::string( $s, $c ) ); diff --git a/bin/zurich/overdue-alert b/bin/zurich/overdue-alert new file mode 100755 index 000000000..fd9c26cb9 --- /dev/null +++ b/bin/zurich/overdue-alert @@ -0,0 +1,83 @@ +#!/usr/bin/env perl + +# zurich/overdue-alert: +# Send email alerts to administrators for overdue admin activities. +# +# Copyright (c) 2012 UK Citizens Online Democracy. All rights reserved. +# Email: matthew@mysociety.org. WWW: http://www.mysociety.org + +use strict; +use warnings; +require 5.8.0; + +use DateTime; +use CronFns; +use FixMyStreet::App; + +my ($verbose, $nomail) = CronFns::options(); + +# Only run on working days +my $now = DateTime->now(); +exit if FixMyStreet::Cobrand::Zurich::is_public_holiday($now) or FixMyStreet::Cobrand::Zurich::is_weekend($now); + +my $cobrand = FixMyStreet::Cobrand->get_class_for_moniker('zurich')->new(); +my %bodies = map { $_->id => $_ } FixMyStreet::App->model("DB::Body")->all; + +loop_through( 'alert-moderation-overdue.txt', 0, 1, [ 'unconfirmed' ] ); +loop_through( 'alert-overdue.txt', 1, 6, 'in progress' ); +loop_through( 'alert-overdue.txt', 0, 6, ['confirmed', 'planned'] ); + +sub loop_through { + my ( $template, $include_parent, $days, $states ) = @_; + my $dtf = FixMyStreet::App->model("DB")->storage->datetime_parser; + my $date_threshold = $dtf->format_datetime(FixMyStreet::Cobrand::Zurich::sub_days( $now, $days )); + + my $reports = FixMyStreet::App->model("DB::Problem")->search( { + state => $states, + created => { '<', $date_threshold }, + bodies_str => { '!=', undef }, + } ); + + my %to_send = (); + while (my $row = $reports->next) { + $to_send{$row->bodies_str} .= '* ' . $row->id . ": '" . $row->title . "'\n\n"; + } + + my $template_path = FixMyStreet->path_to( "templates", "email", "zurich", $template )->stringify; + $template = Utils::read_file( $template_path ); + + foreach my $body_id (keys %to_send) { + send_alert( $template, $body_id, $to_send{$body_id}, $include_parent ); + } +} + +sub send_alert { + my ( $template, $body_id, $data, $include_parent ) = @_; + + my $body = $bodies{$body_id}; + my $body_email = $body->endpoint; + + my $h = { + data => $data, + admin_url => $cobrand->admin_base_url, + }; + + my $to = [ [ $body_email, $body->name ] ]; + if ( $include_parent ) { + my $parent = $body->parent; + my $parent_email = $parent->endpoint; + push @$to, [ $parent_email, $parent->name ]; + } + + FixMyStreet::App->send_email_cron( + { + _template_ => $template, + _parameters_ => $h, + To => $to, + From => [ FixMyStreet->config('CONTACT_EMAIL'), FixMyStreet->config('CONTACT_NAME') ], + }, + FixMyStreet->config('CONTACT_EMAIL'), + $nomail + ); +} + diff --git a/data/kasabi-requirements.txt b/data/kasabi-requirements.txt deleted file mode 100644 index 9b4397f00..000000000 --- a/data/kasabi-requirements.txt +++ /dev/null @@ -1,2 +0,0 @@ -psycopg2 -pytassium diff --git a/perllib/FixMyStreet/DB/Result/Token.pm b/perllib/FixMyStreet/DB/Result/Token.pm index 5525fe7a5..0156af137 100644 --- a/perllib/FixMyStreet/DB/Result/Token.pm +++ b/perllib/FixMyStreet/DB/Result/Token.pm @@ -30,9 +30,6 @@ __PACKAGE__->set_primary_key("scope", "token"); # Created by DBIx::Class::Schema::Loader v0.07017 @ 2012-03-08 17:19:55 # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:+LLZ8P5GXqPetuGyrra2vw -# Trying not to use this -# use mySociety::DBHandle qw(dbh); - use mySociety::AuthToken; =head1 NAME @@ -43,8 +40,6 @@ FixMyStreet::DB::Result::Token Representation of mySociety::AuthToken in the DBIx::Class world. -Mostly done so that we don't need to use mySociety::DBHandle. - The 'data' value is automatically inflated and deflated in the same way that the AuthToken would do it. 'token' is set to a new random value by default and the 'created' timestamp is achieved using the database function diff --git a/perllib/Utils.pm b/perllib/Utils.pm index 8f0ac1820..6d6d5bcde 100644 --- a/perllib/Utils.pm +++ b/perllib/Utils.pm @@ -15,7 +15,6 @@ use strict; use DateTime; use Encode; use File::Slurp qw(); -use mySociety::DBHandle qw(dbh); use mySociety::GeoUtil; use mySociety::Locale; -- cgit v1.2.3 From b203e7907fa4808966521da8e18d9796e28e9b9a Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Thu, 15 Jan 2015 14:02:19 +0000 Subject: Rename test-wrapper to run-tests. As that's what it does; and use it in Travis. --- .travis.yml | 9 +-------- bin/run-tests | 35 +++++++++++++++++++++++++++++++++++ bin/site-specific-install.sh | 2 +- bin/test-wrapper | 35 ----------------------------------- cpanfile | 2 +- perllib/FixMyStreet/TestAppProve.pm | 2 +- 6 files changed, 39 insertions(+), 46 deletions(-) create mode 100755 bin/run-tests delete mode 100755 bin/test-wrapper diff --git a/.travis.yml b/.travis.yml index 1e4b7611f..207f4a3c7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -25,16 +25,9 @@ before_install: install: - .travis/install before_script: - - psql -c 'create database fms;' -U postgres - - psql fms postgres < db/schema.sql - - psql fms postgres < db/alert_types.sql - - psql fms postgres < db/generate_secret.sql - - > - sed -r -e "s,(FMS_DB_USER:) 'fms',\\1 'postgres'," - conf/general.yml-example > conf/general.yml - ./bin/cron-wrapper ./bin/make_po FixMyStreet-EmptyHomes - ./bin/cron-wrapper ./bin/emptyhomes/make_welsh_po - commonlib/bin/gettext-makemo FixMyStreet -script: "bin/cron-wrapper perl /usr/bin/prove -rl t" +script: "bin/run-tests t" after_script: - .travis/after_script diff --git a/bin/run-tests b/bin/run-tests new file mode 100755 index 000000000..42315e2f2 --- /dev/null +++ b/bin/run-tests @@ -0,0 +1,35 @@ +#!/usr/bin/env perl +use strict; use warnings; + +BEGIN { + use FindBin; + require "$FindBin::Bin/../setenv.pl"; +} + +use FixMyStreet::TestAppProve; + +=head1 NAME + +bin/run-tests - spin up a clean database and configuration for tests + +=head1 USAGE + + bin/run-tests t/ # runs prove -r t/ (recursively, saving state) + bin/run-tests --state=failed # runs just failing tests + # delete .prove to reset + + bin/run-tests t/foo.t # runs with prove -v (single test) + +By default we use conf/general.yml-example as the template, and spin up a fresh +Postgres cluster and database. We can override this by doing: + + bin/run-tests --config=general.mycobrand.yml + bin/run-tests --db-config=general.yml # e.g. use your current DB settings + bin/run-tests --all-config=general.yml # e.g. use your current DB settings + +(NB: passing --state=save automagically is experimental, and will be removed if +it turns out to be annoying.) + +=cut + +FixMyStreet::TestAppProve->run(@ARGV); diff --git a/bin/site-specific-install.sh b/bin/site-specific-install.sh index 3d01be469..de75d45ae 100644 --- a/bin/site-specific-install.sh +++ b/bin/site-specific-install.sh @@ -66,4 +66,4 @@ fi echo Installation complete - you should now be able to view the site at: echo http://$HOST/ echo Or you can run the tests by switching to the "'$UNIX_USER'" user and -echo running: $REPOSITORY/bin/cron-wrapper prove -r t +echo running: $REPOSITORY/bin/run-tests t diff --git a/bin/test-wrapper b/bin/test-wrapper deleted file mode 100755 index 9cb1791a1..000000000 --- a/bin/test-wrapper +++ /dev/null @@ -1,35 +0,0 @@ -#!/usr/bin/env perl -use strict; use warnings; - -BEGIN { - use FindBin; - require "$FindBin::Bin/../setenv.pl"; -} - -use FixMyStreet::TestAppProve; - -=head1 NAME - -bin/test-wrapper - spin up a clean database and configuration for tests - -=head1 USAGE - - bin/test-wrapper t/ # runs prove -r t/ (recursively, saving state) - bin/test-wrapper --state=failed # runs just failing tests - # delete .prove to reset - - bin/test-wrapper t/foo.t # runs with prove -v (single test) - -By default we use conf/general.yml-example as the template, and spin up a fresh -Postgres cluster and database. We can override this by doing: - - bin/test-wrapper --config=general.mycobrand.yml - bin/test-wrapper --db-config=general.yml # e.g. use your current DB settings - bin/test-wrapper --all-config=general.yml # e.g. use your current DB settings - -(NB: passing --state=save automagically is experimental, and will be removed if -it turns out to be annoying.) - -=cut - -FixMyStreet::TestAppProve->run(@ARGV); diff --git a/cpanfile b/cpanfile index cda6cc131..e6cb1d83d 100644 --- a/cpanfile +++ b/cpanfile @@ -110,7 +110,7 @@ requires 'File::ChangeNotify'; requires 'Path::Tiny'; requires 'File::Find::Rule'; -feature 'test-wrapper', 'Spin up a test database and config to run tests' => sub { +feature 'run-tests', 'Spin up a test database and config to run tests' => sub { requires 'Test::PostgreSQL'; }; diff --git a/perllib/FixMyStreet/TestAppProve.pm b/perllib/FixMyStreet/TestAppProve.pm index 4d8cdaccb..75e9fe0a4 100644 --- a/perllib/FixMyStreet/TestAppProve.pm +++ b/perllib/FixMyStreet/TestAppProve.pm @@ -14,7 +14,7 @@ FixMyStreet::TestAppProve - spin up a clean database and configuration for tests =head1 USAGE -see bin/test-wrapper for usage +see bin/run-tests for usage =cut -- cgit v1.2.3 From 96dd07e7c1dab2c8f9250af435caad2c2818b5a2 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Thu, 15 Jan 2015 14:18:45 +0000 Subject: Use env in shebang lines, and remove unneeded ones --- bin/emptyhomes/make_welsh_po | 4 +++- bin/geocode | 2 +- bin/gettext-nget-patch | 2 +- bin/handlemail | 3 ++- bin/make_po | 4 +++- bin/open311-update-reports | 2 +- bin/update-schema | 2 +- perllib/CronFns.pm | 2 -- perllib/CrossSell.pm | 2 -- perllib/FixMyStreet/Geocode.pm | 2 -- perllib/FixMyStreet/Geocode/Bing.pm | 2 -- perllib/FixMyStreet/Geocode/FixaMinGata.pm | 2 -- perllib/FixMyStreet/Geocode/Google.pm | 4 +--- perllib/FixMyStreet/Geocode/OSM.pm | 2 -- perllib/FixMyStreet/Geocode/Zurich.pm | 2 -- perllib/FixMyStreet/Map.pm | 2 -- perllib/FixMyStreet/Map/Bing.pm | 2 -- perllib/FixMyStreet/Map/Bromley.pm | 2 -- perllib/FixMyStreet/Map/FMS.pm | 2 -- perllib/FixMyStreet/Map/Google.pm | 2 -- perllib/FixMyStreet/Map/GoogleOL.pm | 2 -- perllib/FixMyStreet/Map/OSM.pm | 2 -- perllib/FixMyStreet/Map/OSM/CycleMap.pm | 2 -- perllib/FixMyStreet/Map/OSM/MapQuest.pm | 2 -- perllib/FixMyStreet/Map/OSM/StreetView.pm | 2 -- perllib/FixMyStreet/Map/Zurich.pm | 2 -- perllib/Geo/Coordinates/CH1903.pm | 2 -- perllib/PoChange.pm | 1 - perllib/Utils.pm | 2 -- 29 files changed, 13 insertions(+), 51 deletions(-) diff --git a/bin/emptyhomes/make_welsh_po b/bin/emptyhomes/make_welsh_po index a85e7c35b..568ccdcde 100755 --- a/bin/emptyhomes/make_welsh_po +++ b/bin/emptyhomes/make_welsh_po @@ -1,5 +1,7 @@ -#!/usr/bin/perl -w +#!/usr/bin/env perl + use strict; +use warnings; # Given a translation of the Empty Homes version of the FixMyStreet .po file, # fix it all up to go direct from FixMyStreet -> EH translation. diff --git a/bin/geocode b/bin/geocode index 254cf1578..2b85ff572 100755 --- a/bin/geocode +++ b/bin/geocode @@ -1,4 +1,4 @@ -#!/usr/bin/perl +#!/usr/bin/env perl =head1 NAME diff --git a/bin/gettext-nget-patch b/bin/gettext-nget-patch index 5ebd8bbcb..92b687552 100755 --- a/bin/gettext-nget-patch +++ b/bin/gettext-nget-patch @@ -1,4 +1,4 @@ -#!/usr/bin/perl +#!/usr/bin/env perl # # xgettext doesn't deal with TT files, but xgettext.pl doesn't find nget()s, sigh. # This will find the nget()s and output a .po file excerpt. diff --git a/bin/handlemail b/bin/handlemail index 8bc016241..5d4ac753c 100755 --- a/bin/handlemail +++ b/bin/handlemail @@ -1,4 +1,4 @@ -#!/usr/bin/perl -w +#!/usr/bin/env perl # # handlemail: # Handle an individual incoming mail message. @@ -14,6 +14,7 @@ my $rcsid = ''; $rcsid .= '$Id: handlemail,v 1.2 2009-02-11 11:04:48 matthew Exp $'; use strict; +use warnings; require 5.8.0; # Horrible boilerplate to set up appropriate library paths. diff --git a/bin/make_po b/bin/make_po index ad1fc1a89..a5b145ecb 100755 --- a/bin/make_po +++ b/bin/make_po @@ -1,5 +1,7 @@ -#!/usr/bin/perl -w +#!/usr/bin/env perl + use strict; +use warnings; # Generates a version of .po file, which is a translation # into a language the same as English, with replacement as specified in PoChange diff --git a/bin/open311-update-reports b/bin/open311-update-reports index 3b77fef89..a964ba8c9 100755 --- a/bin/open311-update-reports +++ b/bin/open311-update-reports @@ -1,4 +1,4 @@ -#!/usr/bin/perl +#!/usr/bin/env perl # # This script utilises the standard Open311 way of getting updates on reports # (by fetching all reports for a body and looking for updates). If possible, diff --git a/bin/update-schema b/bin/update-schema index 30fc1396b..af0da3601 100755 --- a/bin/update-schema +++ b/bin/update-schema @@ -1,4 +1,4 @@ -#!/usr/bin/perl +#!/usr/bin/env perl =head1 NAME diff --git a/perllib/CronFns.pm b/perllib/CronFns.pm index ac7f85596..28268c08e 100755 --- a/perllib/CronFns.pm +++ b/perllib/CronFns.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -w - # CronFns.pm: # Shared functions for cron-run scripts # diff --git a/perllib/CrossSell.pm b/perllib/CrossSell.pm index 0c621e9d0..9567c2b38 100644 --- a/perllib/CrossSell.pm +++ b/perllib/CrossSell.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -w -# # CrossSell.pm: # Adverts from FixMyStreet to another site. # diff --git a/perllib/FixMyStreet/Geocode.pm b/perllib/FixMyStreet/Geocode.pm index 2a318ea5a..b5be152a8 100644 --- a/perllib/FixMyStreet/Geocode.pm +++ b/perllib/FixMyStreet/Geocode.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet::Geocode # The geocoding functions for FixMyStreet. # diff --git a/perllib/FixMyStreet/Geocode/Bing.pm b/perllib/FixMyStreet/Geocode/Bing.pm index 85eef3d0f..702e19814 100644 --- a/perllib/FixMyStreet/Geocode/Bing.pm +++ b/perllib/FixMyStreet/Geocode/Bing.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet::Geocode::Bing # Geocoding with Bing for FixMyStreet. # diff --git a/perllib/FixMyStreet/Geocode/FixaMinGata.pm b/perllib/FixMyStreet/Geocode/FixaMinGata.pm index 2db25f504..2ea92c422 100644 --- a/perllib/FixMyStreet/Geocode/FixaMinGata.pm +++ b/perllib/FixMyStreet/Geocode/FixaMinGata.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Geocode::FixaMinGata # OpenStreetmap forward and reverse geocoding for FixMyStreet. # diff --git a/perllib/FixMyStreet/Geocode/Google.pm b/perllib/FixMyStreet/Geocode/Google.pm index fd65b89b1..11ff8ef80 100644 --- a/perllib/FixMyStreet/Geocode/Google.pm +++ b/perllib/FixMyStreet/Geocode/Google.pm @@ -1,6 +1,4 @@ -#!/usr/bin/perl -# -# FixMyStreet::Geocode +# FixMyStreet::Geocode::Google # The geocoding functions for FixMyStreet. # # Copyright (c) 2010 UK Citizens Online Democracy. All rights reserved. diff --git a/perllib/FixMyStreet/Geocode/OSM.pm b/perllib/FixMyStreet/Geocode/OSM.pm index fd14b0acc..919940f78 100644 --- a/perllib/FixMyStreet/Geocode/OSM.pm +++ b/perllib/FixMyStreet/Geocode/OSM.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Geocode::OSM # OpenStreetmap forward and reverse geocoding for FixMyStreet. # diff --git a/perllib/FixMyStreet/Geocode/Zurich.pm b/perllib/FixMyStreet/Geocode/Zurich.pm index 84fd83fc7..1f0b4fc16 100644 --- a/perllib/FixMyStreet/Geocode/Zurich.pm +++ b/perllib/FixMyStreet/Geocode/Zurich.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet::Geocode::Zurich # Geocoding with Zurich web service. # diff --git a/perllib/FixMyStreet/Map.pm b/perllib/FixMyStreet/Map.pm index f2dd0da6d..7d490fde3 100644 --- a/perllib/FixMyStreet/Map.pm +++ b/perllib/FixMyStreet/Map.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map # Adding the ability to have different maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/Bing.pm b/perllib/FixMyStreet/Map/Bing.pm index 676e70bf6..09c951a5f 100644 --- a/perllib/FixMyStreet/Map/Bing.pm +++ b/perllib/FixMyStreet/Map/Bing.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::Bing # Bing maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/Bromley.pm b/perllib/FixMyStreet/Map/Bromley.pm index 20821236f..fc8726b34 100644 --- a/perllib/FixMyStreet/Map/Bromley.pm +++ b/perllib/FixMyStreet/Map/Bromley.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::Bromley # Bromley have slightly different tiles, with trees etc. # diff --git a/perllib/FixMyStreet/Map/FMS.pm b/perllib/FixMyStreet/Map/FMS.pm index 62849a157..96e265a4d 100644 --- a/perllib/FixMyStreet/Map/FMS.pm +++ b/perllib/FixMyStreet/Map/FMS.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::FMS # Bing and OS StreetView maps on FixMyStreet, using OpenLayers. # diff --git a/perllib/FixMyStreet/Map/Google.pm b/perllib/FixMyStreet/Map/Google.pm index 9deefc033..172d2d60e 100644 --- a/perllib/FixMyStreet/Map/Google.pm +++ b/perllib/FixMyStreet/Map/Google.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::Google # Google maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/GoogleOL.pm b/perllib/FixMyStreet/Map/GoogleOL.pm index 64baf8d36..2dfb697e5 100644 --- a/perllib/FixMyStreet/Map/GoogleOL.pm +++ b/perllib/FixMyStreet/Map/GoogleOL.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::GoogleOL # Google maps on FixMyStreet, using OpenLayers. # diff --git a/perllib/FixMyStreet/Map/OSM.pm b/perllib/FixMyStreet/Map/OSM.pm index 82ddd293b..df2d16b82 100644 --- a/perllib/FixMyStreet/Map/OSM.pm +++ b/perllib/FixMyStreet/Map/OSM.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::OSM # OSM maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/OSM/CycleMap.pm b/perllib/FixMyStreet/Map/OSM/CycleMap.pm index 71b86de8f..8f1de39d2 100644 --- a/perllib/FixMyStreet/Map/OSM/CycleMap.pm +++ b/perllib/FixMyStreet/Map/OSM/CycleMap.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::OSM::CycleMap # OSM CycleMap maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/OSM/MapQuest.pm b/perllib/FixMyStreet/Map/OSM/MapQuest.pm index a7f1b334e..2c3cbaf00 100644 --- a/perllib/FixMyStreet/Map/OSM/MapQuest.pm +++ b/perllib/FixMyStreet/Map/OSM/MapQuest.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::OSM::CycleMap # OSM CycleMap maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/OSM/StreetView.pm b/perllib/FixMyStreet/Map/OSM/StreetView.pm index 8fe4744a4..c70dd93aa 100644 --- a/perllib/FixMyStreet/Map/OSM/StreetView.pm +++ b/perllib/FixMyStreet/Map/OSM/StreetView.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::OSM::StreetView # OSM StreetView maps on FixMyStreet. # diff --git a/perllib/FixMyStreet/Map/Zurich.pm b/perllib/FixMyStreet/Map/Zurich.pm index e09f8c90f..d667a4701 100644 --- a/perllib/FixMyStreet/Map/Zurich.pm +++ b/perllib/FixMyStreet/Map/Zurich.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # FixMyStreet:Map::Zurich # Zurich have their own tileserver. # diff --git a/perllib/Geo/Coordinates/CH1903.pm b/perllib/Geo/Coordinates/CH1903.pm index 612182152..6611af9f2 100644 --- a/perllib/Geo/Coordinates/CH1903.pm +++ b/perllib/Geo/Coordinates/CH1903.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # Geo::Coordinates::CH1903 # Conversion between WGS84 and Swiss CH1903. # diff --git a/perllib/PoChange.pm b/perllib/PoChange.pm index f26161889..1e00c1426 100644 --- a/perllib/PoChange.pm +++ b/perllib/PoChange.pm @@ -1,4 +1,3 @@ -#!/usr/bin/perl -w use strict; package PoChange; diff --git a/perllib/Utils.pm b/perllib/Utils.pm index 6d6d5bcde..7f95d1f5a 100644 --- a/perllib/Utils.pm +++ b/perllib/Utils.pm @@ -1,5 +1,3 @@ -#!/usr/bin/perl -# # Utils.pm: # Various generic utilities for FixMyStreet. # -- cgit v1.2.3 From 8e6d48d5ee351594fc92f1e680ba4a2219ed57ca Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Thu, 15 Jan 2015 15:33:21 +0000 Subject: Remove need for cron-wrapper with existing scripts Call the necessary boilerplate in each script so you can call them directly. Remove boilerplate from files that don't need it. --- .travis.yml | 4 ++-- Vagrantfile | 2 +- bin/cron-wrapper | 6 ++++-- bin/fetch-comments | 7 +++++++ bin/fetch-comments-24hs | 10 +++++++++- bin/geocode | 9 +++++++-- bin/install-as-user | 8 ++++---- bin/make_css_watch | 9 +++++++++ bin/open311-populate-service-list | 8 ++++++++ bin/open311-update-reports | 8 ++++++++ bin/problem-creation-graph | 11 ++++++++--- bin/problems-filed-graph | 11 ++++++++--- bin/run-tests | 6 ++++-- bin/send-alerts | 7 +++++++ bin/send-comments | 7 +++++++ bin/send-questionnaires | 7 +++++++ bin/send-reports | 7 +++++++ bin/update-all-reports | 7 +++++++ bin/update-schema | 16 ++++++++++------ bin/zurich/overdue-alert | 7 +++++++ conf/crontab-example | 24 ++++++++++++------------ conf/sysvinit.example | 2 +- perllib/CronFns.pm | 7 ------- perllib/FixMyStreet/App/Controller/Reports.pm | 2 +- perllib/FixMyStreet/SendReport/London.pm | 1 + script/fixmystreet_app_create.pl | 7 +++++++ script/fixmystreet_app_server.pl | 4 ++++ script/fixmystreet_app_test.pl | 7 +++++++ setenv.pl | 13 ++++++------- t/app/controller/reports.t | 2 +- t/open311.t | 4 ---- t/open311/getservicerequestupdates.t | 4 ---- t/open311/populate-service-list.t | 4 ---- t/utils.t | 4 ---- 34 files changed, 171 insertions(+), 71 deletions(-) diff --git a/.travis.yml b/.travis.yml index 207f4a3c7..2c80b3534 100644 --- a/.travis.yml +++ b/.travis.yml @@ -25,8 +25,8 @@ before_install: install: - .travis/install before_script: - - ./bin/cron-wrapper ./bin/make_po FixMyStreet-EmptyHomes - - ./bin/cron-wrapper ./bin/emptyhomes/make_welsh_po + - ./bin/make_po FixMyStreet-EmptyHomes + - ./bin/emptyhomes/make_welsh_po - commonlib/bin/gettext-makemo FixMyStreet script: "bin/run-tests t" after_script: diff --git a/Vagrantfile b/Vagrantfile index 7cebebbf9..cc6832f9a 100755 --- a/Vagrantfile +++ b/Vagrantfile @@ -38,7 +38,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| echo "****************" echo "You can now ssh into your vagrant box: vagrant ssh" echo "The website code is found in: ~/fixmystreet" - echo "You can run the dev server with: bin/cron-wrapper script/fixmystreet_app_server.pl [-d] [-r] [--fork]" + echo "You can run the dev server with: script/fixmystreet_app_server.pl [-d] [-r] [--fork]" EOS # Create a private network, which allows host-only access to the machine diff --git a/bin/cron-wrapper b/bin/cron-wrapper index 127fd8d8a..c75f0289b 100755 --- a/bin/cron-wrapper +++ b/bin/cron-wrapper @@ -1,8 +1,10 @@ #!/usr/bin/env perl BEGIN { # set all the paths to the perl code - use FindBin; - require "$FindBin::Bin/../setenv.pl"; + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; } exec { $ARGV[0] } @ARGV or die $!; diff --git a/bin/fetch-comments b/bin/fetch-comments index a276433e4..4eb9def14 100755 --- a/bin/fetch-comments +++ b/bin/fetch-comments @@ -8,6 +8,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use FixMyStreet::App; use CronFns; my ($verbose, $nomail) = CronFns::options(); diff --git a/bin/fetch-comments-24hs b/bin/fetch-comments-24hs index 602383d40..27853bcc1 100755 --- a/bin/fetch-comments-24hs +++ b/bin/fetch-comments-24hs @@ -7,9 +7,17 @@ use strict; use warnings; +require 5.8.0; + +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use DateTime; use DateTime::Format::W3CDTF; -require 5.8.0; use FixMyStreet::App; use CronFns; diff --git a/bin/geocode b/bin/geocode index 2b85ff572..2559f7a3c 100755 --- a/bin/geocode +++ b/bin/geocode @@ -6,8 +6,6 @@ geocode - commandline tool to test geocoders =head1 SYNOPSIS - $ eval `perl setenv.pl`, or call with bin/cron-wrapper - $ bin/geocode --geocoder=Bing --cobrand=bromley "Glebe Rd" # ... if your conf/general.yml supplies GEOCODER: @@ -24,6 +22,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Data::Dumper; use Pod::Usage; use feature 'say'; diff --git a/bin/install-as-user b/bin/install-as-user index 33a74c140..0e1c151fb 100755 --- a/bin/install-as-user +++ b/bin/install-as-user @@ -127,15 +127,15 @@ then psql -U "$UNIX_USER" "$DB_NAME" < "$REPOSITORY"/db/alert_types.sql psql -U "$UNIX_USER" "$DB_NAME" < "$REPOSITORY"/db/generate_secret.sql else - bin/cron-wrapper update-schema --commit + bin/update-schema --commit fi echo $DONE_MSG # Generate po and mo files (these invocations taken from Kagee's script): echo "Creating locale .mo files" -bin/cron-wrapper bin/make_po FixMyStreet-EmptyHomes -bin/cron-wrapper bin/emptyhomes/make_welsh_po +bin/make_po FixMyStreet-EmptyHomes +bin/emptyhomes/make_welsh_po commonlib/bin/gettext-makemo FixMyStreet echo $DONE_MSG -bin/cron-wrapper update-all-reports +bin/update-all-reports diff --git a/bin/make_css_watch b/bin/make_css_watch index 63a27a35e..2ae287429 100755 --- a/bin/make_css_watch +++ b/bin/make_css_watch @@ -1,7 +1,16 @@ #!/usr/bin/env perl + use strict; use warnings; use feature 'say'; + +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Cwd qw(abs_path); use File::ChangeNotify; use File::Find::Rule; diff --git a/bin/open311-populate-service-list b/bin/open311-populate-service-list index be1ace3b9..11f9005a5 100755 --- a/bin/open311-populate-service-list +++ b/bin/open311-populate-service-list @@ -2,6 +2,14 @@ use strict; use warnings; + +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use FixMyStreet::App; use Open311::PopulateServiceList; use Getopt::Long::Descriptive; diff --git a/bin/open311-update-reports b/bin/open311-update-reports index a964ba8c9..6a1c894d7 100755 --- a/bin/open311-update-reports +++ b/bin/open311-update-reports @@ -8,6 +8,14 @@ use strict; use warnings; + +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Open311::GetUpdates; use FixMyStreet::App; diff --git a/bin/problem-creation-graph b/bin/problem-creation-graph index e1b6f2cc4..c3d10f934 100755 --- a/bin/problem-creation-graph +++ b/bin/problem-creation-graph @@ -10,13 +10,18 @@ use strict; use warnings; require 5.8.0; -use FindBin; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + chdir "$d/.."; + require "$d/../setenv.pl"; +} + use File::Temp qw(tempfile); use FixMyStreet::App; -chdir("$FindBin::Bin/../"); - my %config = ( gplot_output => "set terminal png font 'Vera.ttf' 9 size 1200,400", extension => '.png', diff --git a/bin/problems-filed-graph b/bin/problems-filed-graph index d3e132f8e..abfe0cf1a 100755 --- a/bin/problems-filed-graph +++ b/bin/problems-filed-graph @@ -10,13 +10,18 @@ use strict; use warnings; require 5.8.0; -use FindBin; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + chdir "$d/.."; + require "$d/../setenv.pl"; +} + use File::Temp qw(tempfile); use FixMyStreet::App; -chdir("$FindBin::Bin/../"); - my %config = ( gplot_output => "set terminal png font 'Vera.ttf' 9 size 1200,600", extension => '.png', diff --git a/bin/run-tests b/bin/run-tests index 42315e2f2..612a13628 100755 --- a/bin/run-tests +++ b/bin/run-tests @@ -2,8 +2,10 @@ use strict; use warnings; BEGIN { - use FindBin; - require "$FindBin::Bin/../setenv.pl"; + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; } use FixMyStreet::TestAppProve; diff --git a/bin/send-alerts b/bin/send-alerts index 00c5426bc..94f7d5509 100755 --- a/bin/send-alerts +++ b/bin/send-alerts @@ -10,6 +10,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use CGI; # XXX use CronFns; diff --git a/bin/send-comments b/bin/send-comments index 2192f674c..2d6b525b5 100755 --- a/bin/send-comments +++ b/bin/send-comments @@ -13,6 +13,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Digest::MD5; use Encode; use CronFns; diff --git a/bin/send-questionnaires b/bin/send-questionnaires index 7ed3c87c2..93d58ed77 100755 --- a/bin/send-questionnaires +++ b/bin/send-questionnaires @@ -10,6 +10,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use CGI; # XXX Awkward kludge use Encode; use CronFns; diff --git a/bin/send-reports b/bin/send-reports index 16296dc5b..3c6f26cdb 100755 --- a/bin/send-reports +++ b/bin/send-reports @@ -10,6 +10,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use CGI; # XXX use CronFns; diff --git a/bin/update-all-reports b/bin/update-all-reports index f4d6113cb..f6cd34210 100755 --- a/bin/update-all-reports +++ b/bin/update-all-reports @@ -10,6 +10,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use FixMyStreet::App; use File::Path (); use File::Slurp; diff --git a/bin/update-schema b/bin/update-schema index af0da3601..ce193f29c 100755 --- a/bin/update-schema +++ b/bin/update-schema @@ -31,9 +31,13 @@ backup of your database before running. use strict; use warnings; -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../commonlib/perllib"; +my $bin_dir; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + $bin_dir = dirname(File::Spec->rel2abs($0)); + require "$bin_dir/../setenv.pl"; +} use mySociety::Config; use mySociety::DBHandle qw(dbh); @@ -41,7 +45,7 @@ use mySociety::MaPit; use Getopt::Long; use Pod::Usage; -mySociety::Config::set_file("$FindBin::Bin/../conf/general"); +mySociety::Config::set_file("$bin_dir/../conf/general"); my %args = ( Name => mySociety::Config::get('FMS_DB_NAME'), User => mySociety::Config::get('FMS_DB_USER'), @@ -115,7 +119,7 @@ if ($upgrade) { if ($version) { die "Not currently implemented"; } - for my $path (glob("$FindBin::Bin/../db/schema_*")) { + for my $path (glob("$bin_dir/../db/schema_*")) { my ($name) = $path =~ /schema_(.*)\.sql$/; next if $name le $current_version; next if $name =~ /$current_version-/; # For number only match @@ -146,7 +150,7 @@ if ($upgrade) { if ($downgrade) { my %downgrades; - for my $path (glob("$FindBin::Bin/../db/downgrade_*")) { + for my $path (glob("$bin_dir/../db/downgrade_*")) { my ($from, $to) = $path =~ /downgrade_(.*)---(.*)\.sql$/; next unless $from eq $current_version; $downgrades{$to} = $path; diff --git a/bin/zurich/overdue-alert b/bin/zurich/overdue-alert index fd9c26cb9..4a8fea830 100755 --- a/bin/zurich/overdue-alert +++ b/bin/zurich/overdue-alert @@ -10,6 +10,13 @@ use strict; use warnings; require 5.8.0; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use DateTime; use CronFns; use FixMyStreet::App; diff --git a/conf/crontab-example b/conf/crontab-example index a29a0de53..68197d7b1 100644 --- a/conf/crontab-example +++ b/conf/crontab-example @@ -11,24 +11,24 @@ PATH=/usr/local/bin:/usr/bin:/bin # send-reports has three rows so that its 8am entry can be run with --verbose to send a morning summary of anything that's gone wrong -5,10,15,20,25,30,35,40,45,50,55 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$FMS/send-reports.lock" "$FMS/bin/cron-wrapper send-reports" || echo "stalled?" -0 0-7,9-23 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-reports.lock" "$FMS/bin/cron-wrapper send-reports" || echo "stalled?" -0 8 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-reports.lock" "$FMS/bin/cron-wrapper send-reports --verbose" || echo "stalled?" +5,10,15,20,25,30,35,40,45,50,55 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$FMS/send-reports.lock" "$FMS/bin/send-reports" || echo "stalled?" +0 0-7,9-23 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-reports.lock" "$FMS/bin/send-reports" || echo "stalled?" +0 8 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-reports.lock" "$FMS/bin/send-reports --verbose" || echo "stalled?" -2 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-alerts.lock" "$FMS/bin/cron-wrapper send-alerts" || echo "stalled?" -22,52 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-questionnaires.lock" "$FMS/bin/cron-wrapper send-questionnaires" || echo "stalled?" +2 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-alerts.lock" "$FMS/bin/send-alerts" || echo "stalled?" +22,52 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-questionnaires.lock" "$FMS/bin/send-questionnaires" || echo "stalled?" # If you utilise Open311 and the updates extension, you will need to run these scripts -#*/5 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-comments.lock" "$FMS/bin/cron-wrapper send-comments" || echo "stalled?" -#*/5 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/fetch-comments.lock" "$FMS/bin/cron-wrapper fetch-comments" || echo "stalled?" +#*/5 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/send-comments.lock" "$FMS/bin/send-comments" || echo "stalled?" +#*/5 * * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/fetch-comments.lock" "$FMS/bin/fetch-comments" || echo "stalled?" -47 0-7,9-23 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/open311-populate-service-list.lock" "$FMS/bin/cron-wrapper open311-populate-service-list" || echo "stalled?" -47 8 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/open311-populate-service-list.lock" "$FMS/bin/cron-wrapper open311-populate-service-list --warn" || echo "stalled?" +47 0-7,9-23 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/open311-populate-service-list.lock" "$FMS/bin/open311-populate-service-list" || echo "stalled?" +47 8 * * * "$FMS/commonlib/bin/run-with-lockfile.sh" -n "$LOCK_DIR/open311-populate-service-list.lock" "$FMS/bin/open311-populate-service-list --warn" || echo "stalled?" # Once an hour, update the all reports stats -13 * * * * "$FMS/bin/cron-wrapper" update-all-reports +13 * * * * "$FMS/bin/update-all-reports" # Once a day on all servers -39 2 * * * "$FMS/bin/cron-wrapper" problems-filed-graph -43 2 * * * "$FMS/bin/cron-wrapper" problem-creation-graph +39 2 * * * "$FMS/bin/problems-filed-graph" +43 2 * * * "$FMS/bin/problem-creation-graph" 00 8 * * * "$FMS/bin/check-for-zombies" $UNIX_USER diff --git a/conf/sysvinit.example b/conf/sysvinit.example index 44424281b..9ad97e6ab 100755 --- a/conf/sysvinit.example +++ b/conf/sysvinit.example @@ -24,7 +24,7 @@ test -f $DAEMON || exit 0 set -e start_daemon() { - su -l -c "cd $SITE_HOME/fixmystreet && bin/cron-wrapper web/fixmystreet_app_fastcgi.cgi -d -l :9000 -n 2" $USER + su -l -c "cd $SITE_HOME/fixmystreet && web/fixmystreet_app_fastcgi.cgi -d -l :9000 -n 2" $USER } stop_daemon() { diff --git a/perllib/CronFns.pm b/perllib/CronFns.pm index 28268c08e..545526c8e 100755 --- a/perllib/CronFns.pm +++ b/perllib/CronFns.pm @@ -3,19 +3,12 @@ # # Copyright (c) 2009 UK Citizens Online Democracy. All rights reserved. # Email: matthew@mysociety.org. WWW: http://www.mysociety.org -# -# $Id: CronFns.pm,v 1.1 2009-07-10 15:17:29 matthew Exp $ package CronFns; use strict; require 5.8.0; -# Horrible boilerplate to set up appropriate library paths. -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - use mySociety::Locale; sub options { diff --git a/perllib/FixMyStreet/App/Controller/Reports.pm b/perllib/FixMyStreet/App/Controller/Reports.pm index 10ef30c90..352c47da8 100644 --- a/perllib/FixMyStreet/App/Controller/Reports.pm +++ b/perllib/FixMyStreet/App/Controller/Reports.pm @@ -74,7 +74,7 @@ sub index : Path : Args(0) { if ($@) { $c->stash->{message} = _("There was a problem showing the All Reports page. Please try again later."); if ($c->config->{STAGING_SITE}) { - $c->stash->{message} .= '

Perhaps the bin/update-all-reports script needs running. Use: bin/cron-wrapper bin/update-all-reports

' + $c->stash->{message} .= '

Perhaps the bin/update-all-reports script needs running. Use: bin/update-all-reports

' . sprintf(_('The error was: %s'), $@); } $c->stash->{template} = 'errors/generic.html'; diff --git a/perllib/FixMyStreet/SendReport/London.pm b/perllib/FixMyStreet/SendReport/London.pm index 2c48a091c..2a1ebc1c3 100644 --- a/perllib/FixMyStreet/SendReport/London.pm +++ b/perllib/FixMyStreet/SendReport/London.pm @@ -5,6 +5,7 @@ use Moose; BEGIN { extends 'FixMyStreet::SendReport'; } use Digest::MD5; +use FindBin; use LWP::UserAgent; use LWP::Simple; diff --git a/script/fixmystreet_app_create.pl b/script/fixmystreet_app_create.pl index c0ec0c898..f43c4ab82 100755 --- a/script/fixmystreet_app_create.pl +++ b/script/fixmystreet_app_create.pl @@ -3,6 +3,13 @@ use strict; use warnings; +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Catalyst::ScriptRunner; Catalyst::ScriptRunner->run('FixMyStreet::App', 'Create'); diff --git a/script/fixmystreet_app_server.pl b/script/fixmystreet_app_server.pl index 3778cf27d..aeb57f0d2 100755 --- a/script/fixmystreet_app_server.pl +++ b/script/fixmystreet_app_server.pl @@ -2,6 +2,10 @@ BEGIN { $ENV{CATALYST_SCRIPT_GEN} = 40; + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; } use Catalyst::ScriptRunner; diff --git a/script/fixmystreet_app_test.pl b/script/fixmystreet_app_test.pl index 4eefd06a4..6f2770730 100755 --- a/script/fixmystreet_app_test.pl +++ b/script/fixmystreet_app_test.pl @@ -1,5 +1,12 @@ #!/usr/bin/env perl +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + my $d = dirname(File::Spec->rel2abs($0)); + require "$d/../setenv.pl"; +} + use Catalyst::ScriptRunner; Catalyst::ScriptRunner->run('FixMyStreet::App', 'Test'); diff --git a/setenv.pl b/setenv.pl index b0cf94a78..39dcc04b0 100755 --- a/setenv.pl +++ b/setenv.pl @@ -5,19 +5,18 @@ use warnings; my $root; -BEGIN { # add the local perllibs too - - # Can't use Path::Class here as we'd load the old debian one. - $root = __FILE__ =~ m{^(.*)/(web|bin)/\.\..*$} ? $1 : `pwd`; - chomp($root); +BEGIN { + use File::Basename qw(dirname); + use File::Spec; + $root = dirname(File::Spec->rel2abs(__FILE__)); } # Set the environment for the FixMyStreet project -# Add the lib/perl5 in perl-external so that we can load local::lib from there +# Add the lib/perl5 in local so that we can load local::lib from there use lib "$root/local/lib/perl5"; -# Add the perl-external dirs properly using local::lib +# Now add the local dir properly using local::lib use local::lib "$root/local"; use lib "$root/commonlib/perllib"; diff --git a/t/app/controller/reports.t b/t/app/controller/reports.t index 1b4fc3526..ecb43f447 100644 --- a/t/app/controller/reports.t +++ b/t/app/controller/reports.t @@ -85,7 +85,7 @@ $fife_problems[10]->update( { }); # Run the cron script that makes the data for /reports so we don't get an error. -system( "bin/cron-wrapper update-all-reports" ); +system( "bin/update-all-reports" ); # check that we can get the page $mech->get_ok('/reports'); diff --git a/t/open311.t b/t/open311.t index 0caacbffc..15bebe2fc 100644 --- a/t/open311.t +++ b/t/open311.t @@ -11,10 +11,6 @@ use HTTP::Response; use DateTime; use DateTime::Format::W3CDTF; -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - use_ok( 'Open311' ); my $o = Open311->new(); diff --git a/t/open311/getservicerequestupdates.t b/t/open311/getservicerequestupdates.t index 00c25a83e..dac10d69b 100644 --- a/t/open311/getservicerequestupdates.t +++ b/t/open311/getservicerequestupdates.t @@ -5,10 +5,6 @@ use warnings; use Test::More; use CGI::Simple; -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - use_ok( 'Open311' ); use_ok( 'Open311::GetServiceRequestUpdates' ); diff --git a/t/open311/populate-service-list.t b/t/open311/populate-service-list.t index d9d65945b..b343b206d 100644 --- a/t/open311/populate-service-list.t +++ b/t/open311/populate-service-list.t @@ -6,10 +6,6 @@ use Test::More; use FixMyStreet::App; -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - use_ok( 'Open311::PopulateServiceList' ); use_ok( 'Open311' ); diff --git a/t/utils.t b/t/utils.t index 8ff9266fd..29759cddc 100644 --- a/t/utils.t +++ b/t/utils.t @@ -4,10 +4,6 @@ use strict; use warnings; use Test::More; -use FindBin; -use lib "$FindBin::Bin/../perllib"; -use lib "$FindBin::Bin/../commonlib/perllib"; - use Utils; my @truncate_tests = ( -- cgit v1.2.3 From f1f9e7fae8df97def1c91f1e5b620ec6ffc1a736 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Thu, 15 Jan 2015 16:13:23 +0000 Subject: Better finding of parent directory when in symlink Fixes #978. --- bin/make_css | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/bin/make_css b/bin/make_css index a8afafb27..3a1fca1a7 100755 --- a/bin/make_css +++ b/bin/make_css @@ -11,13 +11,14 @@ COMPASS=compass SASS=sass -PARENT=$(cd `dirname $0`/../.. && pwd) +DIR=$(cd "$(dirname "$0")" && pwd -P) +PARENT=$(cd "$DIR"/../.. && pwd) if [ -f "$PARENT/gem-bin/compass" ]; then COMPASS=$PARENT/gem-bin/compass SASS=$PARENT/gem-bin/sass fi -DIRECTORY=$(cd `dirname $0`/../web && pwd) +DIRECTORY=$(cd "$DIR"/../web && pwd) DIRS=${@:-`find $DIRECTORY -name "*.scss" -exec dirname {} \; | uniq`} -- cgit v1.2.3 From 683bc0b0898df79a6281ebcab93045b79649484e Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Mon, 19 Jan 2015 16:53:42 +0000 Subject: Use site_name to genericise default submit.txt. --- templates/email/default/submit-brent.txt | 43 ---------------------------- templates/email/default/submit.txt | 15 +++------- templates/email/fixmystreet/submit-brent.txt | 43 ++++++++++++++++++++++++++++ templates/email/fixmystreet/submit.txt | 43 ++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 54 deletions(-) delete mode 100644 templates/email/default/submit-brent.txt create mode 100644 templates/email/fixmystreet/submit-brent.txt create mode 100644 templates/email/fixmystreet/submit.txt diff --git a/templates/email/default/submit-brent.txt b/templates/email/default/submit-brent.txt deleted file mode 100644 index 37adde4df..000000000 --- a/templates/email/default/submit-brent.txt +++ /dev/null @@ -1,43 +0,0 @@ -Subject: FMS Problem Report: - -Dear , - -A user of -FixMyStreet has submitted the following report -of a local problem that they believe might require your attention. - -, or to provide an update on the problem, -please visit the following link: - - - ----------- - -Name: - -Email: - -Subject: - -Details: - -Latitude: - -Longitude: - ----------- - -Replies to this email will go to the user who submitted the problem. - - - -This message was sent via FixMyStreet, a project of UKCOD, registered charity -number 1076346. If there is a more appropriate email address for messages about -, please let us know by visiting . -This will help improve the service for local people. We -also welcome any other feedback you may have. - -FixMyStreet is now available for full integration into council -websites, making life easier for both you and your residents. -Read more here: https://www.fixmystreet.com/council - diff --git a/templates/email/default/submit.txt b/templates/email/default/submit.txt index 315411fb4..0ab29e412 100644 --- a/templates/email/default/submit.txt +++ b/templates/email/default/submit.txt @@ -3,7 +3,7 @@ Subject: Problem Report: Dear , A user of -FixMyStreet has submitted the following report + has submitted the following report of a local problem that they believe might require your attention. , or to provide an update on the problem, @@ -31,13 +31,6 @@ Replies to this email will go to the user who submitted the problem. -This message was sent via FixMyStreet, a project of UKCOD, registered charity -number 1076346. If there is a more appropriate email address for messages about -, please let us know by visiting . -This will help improve the service for local people. We -also welcome any other feedback you may have. - -FixMyStreet is now available for full integration into council -websites, making life easier for both you and your residents. -Read more here: https://www.fixmystreet.com/council - +If there is a more appropriate email address for messages about +, please let us know. This will help improve the +service for local people. We also welcome any other feedback you may have. diff --git a/templates/email/fixmystreet/submit-brent.txt b/templates/email/fixmystreet/submit-brent.txt new file mode 100644 index 000000000..37adde4df --- /dev/null +++ b/templates/email/fixmystreet/submit-brent.txt @@ -0,0 +1,43 @@ +Subject: FMS Problem Report: + +Dear , + +A user of +FixMyStreet has submitted the following report +of a local problem that they believe might require your attention. + +, or to provide an update on the problem, +please visit the following link: + + + +---------- + +Name: + +Email: + +Subject: + +Details: + +Latitude: + +Longitude: + +---------- + +Replies to this email will go to the user who submitted the problem. + + + +This message was sent via FixMyStreet, a project of UKCOD, registered charity +number 1076346. If there is a more appropriate email address for messages about +, please let us know by visiting . +This will help improve the service for local people. We +also welcome any other feedback you may have. + +FixMyStreet is now available for full integration into council +websites, making life easier for both you and your residents. +Read more here: https://www.fixmystreet.com/council + diff --git a/templates/email/fixmystreet/submit.txt b/templates/email/fixmystreet/submit.txt new file mode 100644 index 000000000..315411fb4 --- /dev/null +++ b/templates/email/fixmystreet/submit.txt @@ -0,0 +1,43 @@ +Subject: Problem Report: + +Dear , + +A user of +FixMyStreet has submitted the following report +of a local problem that they believe might require your attention. + +, or to provide an update on the problem, +please visit the following link: + + + +---------- + +Name: + +Email: + +Subject: + +Details: + +Latitude: + +Longitude: + +---------- + +Replies to this email will go to the user who submitted the problem. + + + +This message was sent via FixMyStreet, a project of UKCOD, registered charity +number 1076346. If there is a more appropriate email address for messages about +, please let us know by visiting . +This will help improve the service for local people. We +also welcome any other feedback you may have. + +FixMyStreet is now available for full integration into council +websites, making life easier for both you and your residents. +Read more here: https://www.fixmystreet.com/council + -- cgit v1.2.3 From 311dabb07a166ad44b5c5259358f17ed74a1d1b3 Mon Sep 17 00:00:00 2001 From: Matthew Somerville Date: Thu, 15 Jan 2015 18:10:13 +0000 Subject: Move site_title function to site-name web template This matches the email template behaviour, and makes it much easier to override without requiring a cobrand package file. Fixes #979. --- perllib/FixMyStreet/App.pm | 8 +++---- perllib/FixMyStreet/App/Controller/Report.pm | 3 ++- perllib/FixMyStreet/Cobrand/Default.pm | 8 ------- perllib/FixMyStreet/Cobrand/FixMindelo.pm | 2 -- perllib/FixMyStreet/Cobrand/FixaMinGata.pm | 5 ----- perllib/FixMyStreet/Cobrand/MakeMyIsland.pm | 2 -- perllib/FixMyStreet/Cobrand/SeeSomething.pm | 2 -- perllib/FixMyStreet/Cobrand/ZeroTB.pm | 2 -- perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm | 2 -- templates/web/barnet/header.html | 2 +- templates/web/barnet/site-name.html | 1 + templates/web/base/admin/config_page.html | 5 ++++- templates/web/base/admin/index.html | 2 +- templates/web/base/common_header_tags.html | 2 +- templates/web/base/faq/faq-en-gb.html | 26 +++++++++++------------ templates/web/base/site-name.html | 1 + templates/web/base/static/privacy.html | 2 +- templates/web/bromley/header.html | 2 +- templates/web/bromley/site-name.html | 1 + templates/web/eastsussex/header.html.template | 2 +- templates/web/emptyhomes/header.html | 2 +- templates/web/emptyhomes/site-name.html | 1 + templates/web/fiksgatami/header.html | 2 +- templates/web/fiksgatami/site-name.html | 1 + templates/web/fixamingata/site-name.html | 1 + templates/web/fixmindelo/site-name.html | 1 + templates/web/fixmybarangay/header.html | 2 +- templates/web/fixmybarangay/site-name.html | 1 + templates/web/fixmystreet/header.html | 2 +- templates/web/fixmystreet/header_logo.html | 2 +- templates/web/fixmystreet/header_opengraph.html | 2 +- templates/web/harrogate/header.html | 2 +- templates/web/harrogate/static/privacy.html | 2 +- templates/web/hart/header.html | 12 ++--------- templates/web/makemyisland/header_logo.html | 2 +- templates/web/makemyisland/site-name.html | 1 + templates/web/oxfordshire/header.html | 2 +- templates/web/seesomething/site-name.html | 1 + templates/web/stevenage/header.html | 2 +- templates/web/stevenage/site-name.html | 1 + templates/web/warwickshire/header.html | 2 +- templates/web/zerotb/site-name.html | 1 + templates/web/zurich/header.html | 2 +- templates/web/zurich/site-name.html | 1 + 44 files changed, 57 insertions(+), 71 deletions(-) create mode 100644 templates/web/barnet/site-name.html create mode 100644 templates/web/base/site-name.html create mode 100644 templates/web/bromley/site-name.html create mode 100644 templates/web/emptyhomes/site-name.html create mode 100644 templates/web/fiksgatami/site-name.html create mode 100644 templates/web/fixamingata/site-name.html create mode 100644 templates/web/fixmindelo/site-name.html create mode 100644 templates/web/fixmybarangay/site-name.html create mode 100644 templates/web/makemyisland/site-name.html create mode 100644 templates/web/seesomething/site-name.html create mode 100644 templates/web/stevenage/site-name.html create mode 100644 templates/web/zerotb/site-name.html create mode 100644 templates/web/zurich/site-name.html diff --git a/perllib/FixMyStreet/App.pm b/perllib/FixMyStreet/App.pm index fd70ffda8..769a6bb8f 100644 --- a/perllib/FixMyStreet/App.pm +++ b/perllib/FixMyStreet/App.pm @@ -12,6 +12,7 @@ use Memcached; use mySociety::Email; use mySociety::Random qw(random_bytes); use FixMyStreet::Map; +use Utils; use Path::Class; use URI; @@ -388,10 +389,9 @@ sub send_email_cron { $sig = Encode::decode('utf8', $sig); $params->{_parameters_}->{signature} = $sig; - $tt->process( 'site_name.txt', $params, \$site_name ); - $site_name = Encode::decode('utf8', $site_name); - my $site_title = $cobrand ? $cobrand->site_title : ''; - $params->{_parameters_}->{site_name} = $site_name || $site_title; + $tt->process( 'site-name.txt', $params, \$site_name ); + $site_name = Utils::trim_text(Encode::decode('utf8', $site_name)); + $params->{_parameters_}->{site_name} = $site_name; $params->{_line_indent} = ''; my $email = mySociety::Locale::in_gb_locale { mySociety::Email::construct_email($params) }; diff --git a/perllib/FixMyStreet/App/Controller/Report.pm b/perllib/FixMyStreet/App/Controller/Report.pm index c20a6754a..88a49f6c9 100644 --- a/perllib/FixMyStreet/App/Controller/Report.pm +++ b/perllib/FixMyStreet/App/Controller/Report.pm @@ -112,9 +112,10 @@ sub load_problem_or_display_error : Private { ); } elsif ( $problem->non_public ) { if ( !$c->user || $c->user->id != $problem->user->id ) { + my $site_name = Utils::trim_text($c->render_fragment('site-name.html')); $c->detach( '/page_error_403_access_denied', - [ sprintf(_('That report cannot be viewed on %s.'), $c->cobrand->site_title) ] # + [ sprintf(_('That report cannot be viewed on %s.'), $site_name) ] # ); } } diff --git a/perllib/FixMyStreet/Cobrand/Default.pm b/perllib/FixMyStreet/Cobrand/Default.pm index ec7a8a81d..7f1fba67a 100644 --- a/perllib/FixMyStreet/Cobrand/Default.pm +++ b/perllib/FixMyStreet/Cobrand/Default.pm @@ -146,14 +146,6 @@ Can be specified in template. sub enter_postcode_text { } -=head2 site_title - -The name of the site - -=cut - -sub site_title { return 'FixMyStreet'; } - =head2 set_lang_and_domain my $set_lang = $cobrand->set_lang_and_domain( $lang, $unicode, $dir ) diff --git a/perllib/FixMyStreet/Cobrand/FixMindelo.pm b/perllib/FixMyStreet/Cobrand/FixMindelo.pm index fd3a55c6c..59debf157 100644 --- a/perllib/FixMyStreet/Cobrand/FixMindelo.pm +++ b/perllib/FixMyStreet/Cobrand/FixMindelo.pm @@ -4,8 +4,6 @@ use base 'FixMyStreet::Cobrand::Default'; use strict; use warnings; -sub site_title { return 'FixMindelo'; } - sub country { return 'CV'; } diff --git a/perllib/FixMyStreet/Cobrand/FixaMinGata.pm b/perllib/FixMyStreet/Cobrand/FixaMinGata.pm index 98c157aef..60f98dd47 100644 --- a/perllib/FixMyStreet/Cobrand/FixaMinGata.pm +++ b/perllib/FixMyStreet/Cobrand/FixaMinGata.pm @@ -9,11 +9,6 @@ use mySociety::MaPit; use FixMyStreet::Geocode::FixaMinGata; use DateTime; -sub site_title { - my ($self) = @_; - return 'FixaMinGata'; -} - sub country { return 'SE'; } diff --git a/perllib/FixMyStreet/Cobrand/MakeMyIsland.pm b/perllib/FixMyStreet/Cobrand/MakeMyIsland.pm index ebeb15fbb..f263df4cc 100644 --- a/perllib/FixMyStreet/Cobrand/MakeMyIsland.pm +++ b/perllib/FixMyStreet/Cobrand/MakeMyIsland.pm @@ -4,8 +4,6 @@ use base 'FixMyStreet::Cobrand::Default'; use strict; use warnings; -sub site_title { return 'MakeMyIsland'; } - sub country { return 'MV'; } diff --git a/perllib/FixMyStreet/Cobrand/SeeSomething.pm b/perllib/FixMyStreet/Cobrand/SeeSomething.pm index 5e6d3a8cd..775ba770b 100644 --- a/perllib/FixMyStreet/Cobrand/SeeSomething.pm +++ b/perllib/FixMyStreet/Cobrand/SeeSomething.pm @@ -9,8 +9,6 @@ sub council_area { return 'West Midlands'; } sub council_name { return 'See Something Say Something'; } sub council_url { return 'seesomething'; } sub area_types { [ 'MTD' ] } -sub site_title { return 'See Something, Say Something'; } - sub site_restriction { my $self = shift; diff --git a/perllib/FixMyStreet/Cobrand/ZeroTB.pm b/perllib/FixMyStreet/Cobrand/ZeroTB.pm index 087bf0912..cdf4e5ad4 100644 --- a/perllib/FixMyStreet/Cobrand/ZeroTB.pm +++ b/perllib/FixMyStreet/Cobrand/ZeroTB.pm @@ -4,8 +4,6 @@ use base 'FixMyStreet::Cobrand::Default'; use strict; use warnings; -sub site_title { return 'ZeroTB'; } - sub enter_postcode_text { return _ ('Enter a nearby street name and area, postal code or district in Delhi'); } sub country { diff --git a/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm b/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm index 646967bfc..63a91697d 100644 --- a/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm +++ b/perllib/FixMyStreet/DB/ResultSet/Questionnaire.pm @@ -92,8 +92,6 @@ sub send_questionnaires_period { . $row->user->email . "\n" if $params->{verbose}; - $h{site_name} = $cobrand->site_title(); - my $result = FixMyStreet::App->send_email_cron( { _template_ => $template, diff --git a/templates/web/barnet/header.html b/templates/web/barnet/header.html index 7b296c208..0e9bbfa42 100644 --- a/templates/web/barnet/header.html +++ b/templates/web/barnet/header.html @@ -30,7 +30,7 @@ - [% INCLUDE 'common_header_tags.html', js_override = '/cobrands/fixmystreet/fixmystreet.js', site_title = 'Barnet Council FixMyStreet' %] + [% INCLUDE 'common_header_tags.html', js_override = '/cobrands/fixmystreet/fixmystreet.js' %] diff --git a/templates/web/barnet/site-name.html b/templates/web/barnet/site-name.html new file mode 100644 index 000000000..78563bdf3 --- /dev/null +++ b/templates/web/barnet/site-name.html @@ -0,0 +1 @@ +Barnet Council FixMyStreet diff --git a/templates/web/base/admin/config_page.html b/templates/web/base/admin/config_page.html index 0b29590e4..211d41f31 100644 --- a/templates/web/base/admin/config_page.html +++ b/templates/web/base/admin/config_page.html @@ -39,7 +39,10 @@ running version [% git_version || 'unknown' %]. [% INCLUDE with_cobrand value="ADMIN_BASE_URL" cob=c.cobrand.admin_base_url %] [% INCLUDE subsection heading="Display" %] -[% INCLUDE just_value value="Site title" conf = c.cobrand.site_title %] + + Site title + [% PROCESS 'site-name.html' -%] + [% allowed_conf = FOR k IN c.config.ALLOWED_COBRANDS %] [% IF k.keys %][% k.items.join(':') %][% ELSE %][% k %][% END %] [%- ',' IF NOT loop.last %] diff --git a/templates/web/base/admin/index.html b/templates/web/base/admin/index.html index 17f7bd22a..c9d2b8deb 100644 --- a/templates/web/base/admin/index.html +++ b/templates/web/base/admin/index.html @@ -1,7 +1,7 @@ [% INCLUDE 'admin/header.html' title=loc('Summary') -%]

-This is the administration interface for [% c.cobrand.site_title %]. If you +This is the administration interface for [% PROCESS 'site-name.html' -%]. If you need any help or guidance, there is plenty of online documentation. The FixMyStreet Platform is actively supported by diff --git a/templates/web/base/common_header_tags.html b/templates/web/base/common_header_tags.html index 49b649eca..16098b7f5 100644 --- a/templates/web/base/common_header_tags.html +++ b/templates/web/base/common_header_tags.html @@ -42,5 +42,5 @@ [% "$title :: " | html IF title %] - [% site_title || 'FixMyStreet' %] + [% PROCESS 'site-name.html' -%] diff --git a/templates/web/base/faq/faq-en-gb.html b/templates/web/base/faq/faq-en-gb.html index 55046ce4a..e1e07a8d7 100755 --- a/templates/web/base/faq/faq-en-gb.html +++ b/templates/web/base/faq/faq-en-gb.html @@ -5,7 +5,7 @@
@@ -15,15 +15,15 @@
What is this site?
-
[% c.cobrand.site_title %] is a site to help people report to their +
[% PROCESS 'site-name.html' -%] is a site to help people report to their local authority, view, or discuss local issues they’ve found, by simply locating them on a map.
-
How do I get in touch with [% c.cobrand.site_title %]?
+
How do I get in touch with [% PROCESS 'site-name.html' -%]?
Here’s our contact page.
-
What sort of problems should I report with [% c.cobrand.site_title %]?
-
This depends upon the precise purpose of [% c.cobrand.site_title %]; +
What sort of problems should I report with [% PROCESS 'site-name.html' -%]?
+
This depends upon the precise purpose of [% PROCESS 'site-name.html' -%]; in the UK, FixMyStreet is primarily for reporting things which are broken or dirty or damaged or dumped, and need fixing, cleaning or clearing, such as: @@ -38,8 +38,8 @@